texosquery

Cross-platform Java application to query OS information designed for use in TeX's shell escape mechanism.

Licence

This material is subject to the LaTeX Project Public License. See http://www.ctan.org/license/lppl1.3 for the details of that license.

Copyright 2016-2020 Nicola Talbot

Description

The application can query the following:

• locale information
• default file encoding
• current working directory
• user home directory
• temporary directory
• OS name, arch and version
• Current date and time in PDF format (for TeX formats that don't provide \pdfcreationdate)
• Date-time stamp of a file in PDF format (for TeX formats that don't provide \pdffilemoddate)
• Size of a file in bytes (for TeX formats that don't provide \pdffilesize)
• Contents of a directory (captured as a list)
• Directory contents filtered by regular expression (captured as a list)
• URI of a file
• Canonical path of a file

All paths use a forward slash as directory divider so results can be used, for example, in commands like \includegraphics.

There are files provided for easy access in (La)TeX documents:

• texosquery.tex : generic TeX code
• texosquery.sty : LaTeX package

This provides the command \TeXOSQuery to run texosquery using TeX's shell escape mechanism and capture the result in a control sequence. The category code of most of TeX's default special characters (and some other potentially problematic characters) is temporarily changed to 12 while reading the result. Some of the texosquery output contains short markup commands (such as \wrp) which are internally converted within \TeXOSQuery.

Important Notes

The TeX code uses a piped shell escape to capture the result from texosquery. This means that you must have the piped shell escape enabled to make use of this feature. MiKTeX users need the --enable-pipes option on the TeX command line to enable this. The alternative is to run texosquery outside of TeX and capture the output in a temporary file, which can then be read using \TeXOSQueryFromFile.

You need to correctly set up the configuration file texosquery.cfg to match your system. The TeX package managers can't do this for you automatically. Copy the file to either your TEXMFHOME or TEXMFLOCAL tree to prevent it from being overwritten by subsequent updates.

texosquery-jre8 is on the restricted list for TeX Live 2017. To take advantage of this, you must have at least Java 8 installed and edit the texosquery.cfg file to:

\def\TeXOSInvokerName{texosquery-jre8}
\TeXOSQueryAllowRestricted

(Note that the second line above has been uncommented.)

Note that due to TeX's security measures, quotes can't be used in the restricted mode's shell escape. This means that if you need to pass a file name containing a space as an argument to texosquery, you'll have to use the unrestricted mode.

Installation

Installation is best done using your TeX package manager. Manual installation instructions are described below. For more detail, see the documentation.

Compiling the Documentation

To compile the documentation:

pdflatex texosquery.dtx
makeglossaries texosquery
makeindex -s gglo.ist -t texosquery.glg -o texosquery.gls texosquery.glo
pdflatex texosquery.dtx
makeindex -s gind.ist texosquery.idx
pdflatex texosquery.dtx
pdflatex texosquery.dtx

Extracting the Files

Except for the .jar files, the texosquery files are all bundled inside texosquery.dtx with the extraction commands provided in texosquery.ins. To extract all the files do:

tex texosquery.ins

The .sh files are bash scripts for Unix-like users. These will need to be set to executable using chmod. I recommend that you also remove the .sh extension. Windows users can deleted these files.

The .batch files are batch scripts for Windows users. Unix-like users can delete these files. These files are given the extension .batch as TeX on Windows doesn't allow the creation of .bat files. The extension will need to be changed to .bat after the files have been extracted.

Installing the Application

In the following, replace TEXMF with the path to your TEXMF directory. You can find your home TEXMF directory using

kpsewhich -var-value=TEXMFHOME

There are now three Java applications provided by this package:

• texosquery.jar (requires at least Java 7)
• texosquery-jre8.jar (requires at least Java 8)
• texosquery-jre5.jar (requires at least Java 5)

(See the "Security" section below.)

There are corresponding bash scripts for Unix-like users (the .sh extension added by texosquery.ins should be removed and the files made executable):

• texosquery
• texosquery-jre8
• texosquery-jre5

There are also corresponding batch scripts for Windows users (the .batch extension created by the .ins file should be changed to .bat):

• texosquery.bat
• texosquery-jre8.bat
• texosquery-jre5.bat

Each script uses kpsewhich to find the corresponding .jar file and runs it.

Put the .jar files in TEXMF/scripts/texosquery/ and the bash or .bat files somewhere on your path. You may or may not need to refresh the TeX database.

To test the installation, run

texosquery -v

in the command prompt or terminal. (Also try with texosquery-jre8 and texosquery-jre5.)

If successful, it should show the version number.

Installing the TeX Code

The .tex, .cfg and .sty files should all be extracted from texosquery.dtx with tex texosquery.ins as described above.

• Move texosquery.tex to TEXMF/tex/generic/texosquery/
• Move texosquery.cfg to TEXMF/tex/generic/texosquery/
• Move texosquery.sty to TEXMF/tex/latex/texosquery/
• Move texosquery.pdf to TEXMF/doc/generic/texosquery/

The texosquery.cfg file allows you to specify which application you want to use. First check which version of the Java Runtime Environment (JRE) you have installed:

java -version

This should display the version information. (For example, "1.8.0_92")

If the version number starts with 1.8 ("Java 8"), then you can use texosquery-jre8.jar. This is the full application. You can use texosquery or texosquery-jre5 as well, but there's less locale support with them. The texosquery-jre8 bash script and texosquery-jre8.bat batch script invokes Java with -Djava.locale.providers=CLDR,JRE.

If Windows users have simply been supplied with an executable versions of the texosquery-jre8.jar file (texosquery-jre8.exe) then you'll have to set the JAVA_TOOL_OPTIONS environment variable instead.

This java.locale.providers setting allows Java to access locale information from the Unicode Common Locale Data Repository (CLDR) installed on their system. The pending Java 9 should include this by default, so this will only be relevant to Java 8. Earlier versions of Java (7 or below) don't have this option, which limits the locale support to that which is natively provided by the JRE. Note that Java 7 and earlier versions have reached their end of life and are now deprecated.

If the version information starts with 1.7 ("Java 7"), then you can use texosquery.jar. This is the default application and provides most of the functions of the full application, but there's less locale support. You can also use the even more limited texosquery-jre5, but you can't use texosquery-jre8, so you can't take advantage of the CLDR.

If the version information starts with 1.5 ("Java 5") or 1.6 ("Java 6"), then you can only use texosquery-jre5. The locale support is significantly reduced in this case and there's no support for language scripts.

Once you have determined which application you want to use, edit the texosquery.cfg so that \TeXOSInvokerName is defined to the appropriate invocation. For example, with Java 8:

\def\TeXOSInvokerName{texosquery-jre8}

or with Java 5 or 6:

\def\TeXOSInvokerName{texosquery-jre5}

(bash users will have to add the .sh extension if it hasn't been removed, as per the above instructions.)

Examples:

Plain TeX:

% arara: pdftex: {shell: on}
\input texosquery

\TeXOSQuery{\result}{-b}
IEFT BCP 47 language tag: \result.

\TeXOSQueryLocale{\result}
POSIX locale: \result.

\TeXOSQueryCwd{\result}
cwd: {\tt \result}.

\TeXOSQueryNow{\result}
now: \result.

\TeXOSQueryFileDate{\result}{\jobname.tex}
file date: \result.

\TeXOSQueryFileSize{\result}{\jobname.tex}
file size: \result bytes.

\bye

LaTeX:

% arara: pdflatex: {shell: on}
\documentclass{article}

\usepackage{texosquery}
\usepackage{etoolbox}

\begin{document}

\TeXOSQuery{\result}{-b}
IEFT BCP 47 language tag: \result.

\TeXOSQueryLocale{\result}
POSIX locale: \result.

\TeXOSQueryCwd{\result}
cwd: \texttt{\result}.

\TeXOSQueryNow{\result}
now: \result.

\TeXOSQueryFileDate{\result}{\jobname.tex}
file date: \result.

\TeXOSQueryFileSize{\result}{\jobname.tex}
file size: \result bytes.

\TeXOSQueryFilterFileList{\result}{,}{.+\string\.(png|jpg)}{.}
All jpg and png files in current directory:

\renewcommand{\do}[1]{\texttt{#1}.\par}
\expandafter\docsvlist\expandafter{\result}

\end{document}

For a full list of available commands, see the documentation (texosquery.pdf).

Troubleshooting

If something goes wrong with the call to texosquery, the control sequence will be set to empty. If this happens, here are the steps to diagnose the problem:

1. Check the log file for any lines starting with TeXOSQuery: For example: TeXOSQuery: texosquery-jre8 --pdfnow If found, this means that the dry run mode was on and the shell escape wasn't used. Ensure that the shell escape is enabled when you build the document.

2. If the log file contains (|texosquery options) then the shell escape was used but texosquery returned an empty value. In this case, copy the part between (| and ) and paste it into a command prompt or terminal but insert --debug at the start of the options list. For example, if the log file contains (|texosquery-jre8 --cwd) then run

texosquery-jre8 --debug --cwd

This should now display an error message explaining the problem. (For example, read access forbidden or file not found or a security exception.)

Security

As from version 1.2, all variants of texosquery (JRE5, 7 and 8) obey TeX Live's openin_any setting. Any of the actions that involve reading file information won't work if read access is forbidden by openin_any or by the operating system. MiKTeX doesn't use the openin_any setting, so if not set texosquery will assume a (any).

In addition to obeying openin_any, the file listing actions (such as --list) for the JRE7 and 8 variants also prohibit listing the contents outside of the current working directory's path even if openin_any=a. This means that you can't, for example, list the contents of .. (the current working directory's parent) nor can you try walking the entire file system. The --walk action additionally won't descend hidden directories. This extra restriction is designed to prevent malicious code from trying to use texosquery to look around your filing system.

The Java 5 version texosquery-jre5 is the least secure and doesn't have this additional restriction on the file listing actions. However, it still obeys openin_any. The --walk action is not available with texosquery-jre5.

Examples:

texosquery-jre8 --debug --walk ',' '.*' /

This action is forbidden:

texosquery-jre8: Can't walk directory: /

Changing / to . is allowed, but not for texosquery-jre5:

texosquery-jre5 --debug --walk ',' '.*' .

returns:

texosquery-jre5: walk requires at least JRE 7 version

With openin_any=a, the --userhome action is usually allowed (depends on the operating system and Java's security manager).

texosquery-jre8 --debug --userhome

returns

\fslh home\fslh nlct

(which \TeXOSQuery converts to /home/nlct.)

However, unless this also happens to be the current working directory, it's not possible to obtain a file listing:

texosquery-jre8 --debug --list ',' /home/nlct

returns

texosquery-jre8: Unable to list contents of: /home/nlct
texosquery-jre8: Listing outside cwd path not permitted: /home/nlct

Source code

The Java source is in the .java files and the manifest for each .jar file is Manifest*.txt (Manifest-jre8.txt for texosquery-jre8 etc). Assuming the following directory structure:

java/TeXOSQuery.java
java/TeXOSQueryJRE5.java
java/TeXOSQueryJRE7.java
java/TeXOSQueryJRE8.java
java/QueryAction.java
java/QueryActionType.java
java/FileSortType.java
java/FileSortComparator.java
java/FileListType.java
java/FileWalkVisitor.java
java/Manifest-jre5.txt
java/Manifest-jre7.txt
java/Manifest-jre8.txt
classes/com/dickimawbooks/texosquery/

Then to create texosquery-jre8.jar, do (for JDK version 1.8):

cd java 
javac -d ../classes TeXOSQuery.java QueryAction.java QueryActionType.java TeXOSQueryJRE8.java FileSortType.java FileSortComparator.java FileListType.java FileWalkVisitor.java
cd ../classes
jar cmf ../java/Manifest-jre8.txt ../texosquery-jre8.jar com/dickimawbooks/texosquery/*.class

Similarly for the other .jar files, replacing 8 with 7 or 5. (The Java 7 version should really be texosquery-jre7.jar, but is simply named texosquery.jar in the distribution for backward compatibility reasons. If you compile the code, the .jar name is your choice.)

---

Source on GitHub: https://github.com/nlct/texosquery

Author Home Page: http://www.dickimaw-books.com/