william.famille-blum.org news

On text sorting, the fact that Notepad++ is the best text editor, and 2011 wishes!

A few years ago I spent some time looking for the best text editor for Windows. After testing pretty much all the editors available at the time (Notepad2, Notepad++, UltraEdit, UltraPad, TextPad �) I finally reckoned that the most powerful and versatile one was Notepad++. I have been using it since then and I regularly recommend it to other people.
I could not convince my father to ditch TextPad though because of its superior text sorting feature. It's true that the standard sorting plugin shipping with Notepad++ is pretty basic. It only lets you sort the file at a given column and by lexicographical order only.
TextPad on the other hand lets you define a sort criterion based on up to three user-defined columns. Also for each column the comparison can be either numerical or lexicographical.

The thing is that I am very stubborn: Notepad++ *is* the best editor for Windows, and so to prove it to my father there were no way round it: I had to implement the missing sorting features for Notepad++!
Because Notepad++ is open-source I was able to start from existing code. It thus took less than a day to implement the missing feature as a plugin! Of course it had to be superior than TextPad's implementation. Hence it lets you define an unlimited number of columns, unlike TexPad which is limited to only three.

I figured out that my father may not be the only one interested by the plugin so here you go: You can download the plugin including sources at NppColumnSort.zip

To install it just copy the plugin DLL (under 'Unicode Release' in the zip file) to the Notepad++ plugin directory (%ProgramFiles(x86)%\Notepad++\plugins if your Windows is 64bit; %ProgramFiles%\Notepad++\plugins if your Windows is 32bit).

I take the opportunity to wish you a great year 2011 and hope that all your problems will be sorted in whichever order you prefer be it lexicographical or numerical!

Plotting thesis word count

I have been playing around with PowerShell lately. Here is the little exercise that I set myself: measure and plot the evolution of the number of words in a TeX document.

The script takes a list of folders as parameters. For every TeX file found it calls texcount.pl to counts the number of words in the document. The total number of words is then saved to a CSV file and two graphs are plotted: one using GNU plot and the other one using the Google Visualization API.

I did not have this script at the time of my studies, but since I have been using Subversion to backup and manage revisions of my writings throughout my studies I was able to retrieve past revisions of my work and generate a few word count measurements. The graph below, generated using the PowerShell script, shows the evolution of word count in my D.Phil thesis.

The graph reveals few facts to the reader. For instance that I was not very productive during the first year... But it also shows that I started writing up early on (thanks to my supervisor's advice). Some doctoral students prefer to postpone writing up till the last few months. I am glad I did not adopt this strategy; it made the whole writing process very smooth and eliminate a lot of stress towards the end.

The graph also shows two particularly productive periods corresponding to two important milestones in the D.Phil process at Oxford: the transfer and the confirmation. The thesis reached a plateau on October 5th 2008, the date when I submitted the thesis to the examiners. There were subsequent minor modifications requested by the examiners after the viva, as well as some cosmetic changes for the camera-ready version sent to the Bodleian.

Here is the script in case you are interested: twc.ps1. I suggest you to use it to monitor your progress on your writings by setting it as a daily job under the Windows Task Scheduler.

The usage syntax is
./twc.ps1 [-tag name] [-date date] [-output outputdir] inputdirs

For example the command
./twc.ps1 -tag first_version -date '16 Nov 2010' -output c:\report c:\thesis
creates a datapoint named 'first_version' at the specified date whose value is the total number of words in TeX documents under directory c:\thesis; the CSV file and graphs are stored under c:\report.

For more examples just run the command without argument.

SyncTeX-related news

- Most TeX distributions (TeXLive 2008 and MikTex 2.7) have now been updated to support SyncTeX. The option is activated with the -synctex switch at the command-line as follows:
pdflatex -synctex=-1 mybook.tex

This command produces a file name 'mybook.synctex' which can then be used to perform PDF<->TEX synchronization. The file can be large so you can instead produce a compressed version (mybook.synctex.gz) with:
pdflatex -synctex=1 mybook.tex

- The SyncTeX patch that I've developed is now maintained in the main development branch of SumatraPDF; it is part of the latest official release (version 0.9.3).

I will still continue to publish my own builds of SumatraPDF to my website (the executable is updated regularly even though I am not posting about it): at the moment the only difference with the official release is the presence of a dialog box to let the user configure the inverse-search command. (In the official release this is done by passing command line parameters to SumatraPDF.exe.)

- I wrote a short documentation to help you configure your favourite editors for synchronization with SumatraPDF.

Links:
- Editors configuration
- My own build of SumatraPDF
- SumatraPDF official build
- MikTeX and TeXLive

SyncTeX and SumatraPDF

The author of the pdfsync TeX package, J�r�me Laurens, has developed a new technology called SyncTex that will eventually replace pdfsync. SyncTex is directly integrated in pdftex. This has many benefits: there is no need to load a special package in your .tex document, there is no more incompatibilities with latex packages, and the synchronization is more precise. One disadvantage is that the generated synchronization file can be huge, and for that reason it has an option to zip it.

Installing SyncTex

SyncTex is not yet released in current TeX distributions (TeXLive 2007), but you can install it by following these steps:
1- install the TexLive distribution
2- replace the following three files:
C:\TeXLive2007\bin\win32\pdftex.exe
C:\TeXLive2007\bin\win32\pdftex.dll
C:\TeXLive2007\bin\win32\kpathsea356.dll
by pdftex.dll, pdftex.exe, and kpathsea356.dll.
3- Regenerate the format files using the setup program from the TexLive CD

Using SyncTex

Now to generate .synctex files you just need to specify the --synctex command-line argument to pdftex as follows:
pdflatex --synctex=-1 test.tex

Synchronizing with SumatraPDF

The author has provided a reference parser implementation for synchronization files generated by SyncTex. I have integrated it in SumatraPDF. If people are interested to test it, a binary version is available here.

I plan to commit my changes to the Sumatra code base in the coming days if I see that it works well with me.

PDF-Latex synchronization (continued)

Update 21 May 2008: script file for emacs added

This is the dual of the previous post: I have now implemented forward-search in SumatraPDF, i.e. the ability to go from the source .tex file to the corresponding location in the PDF. The communication between the TeX editor and SumatraPDF relies on the DDE protocol. (Most TeX editors such as WinEdt and TeXnicCenter support it.)

Binary

If you want to give it a try, the updated binary can be downloaded here.

Configuring with WinEdt

For WinEdt, I have created a macro file to do all the stuff automatically. To install it, proceed as follows:

1- Download the macro file Sumatra-Find.edt.
2- Save it in the folder: C:\Program Files\WinEdt Team\WinEdt\Exec\
3- Edit the file: goto line 22 and replace the string 'd:\Tools\SumatraPDF-sync.exe' by the path to your local copy of Sumatra.
4- Go to the menu 'Option\Menu setup'. Create a new macro menu somewhere (under 'Accessories' for instance) and fill the options as follows:
Name: Sumatra Find
Macro: Exe('%b\Exec\Sumatra-Find.edt');
Requires File Filter: %P\%N.pdf
Start in: %P
Shortcut: F8

That's it! Now you can just press F8 anywhere in your .tex sources files and it will automatically locate the corresponding position in the PDF.

Configuring with TeXnicCenter

To setup forward-search with TeXnicCenter proceed as follows:
1- Menu 'Build\Define output profiles...'
2- Select the profile "Latex=>PDF"
3- Select the tab 'Viewer'
4- Fill the options as follows:
Path of executable: c:\tools\SumatraPDF-sync.exe
Forward search: select DDE command
Command: [ForwardSearch("%bm.pdf","%nc",%l,0)]
server: SUMATRA
topic: control

Configuring with ntEmacs

1- Download the script file sumatra-forward.el and load it from your init.el file.
2- Make sure that you have the ddeclient tools. (It is bundled with ntemacs)
3- Press F8 in the TeX document to trigger a forward-search

Synchronization in action

Screenshot

Enjoy! And feel free to post a comment to let me know how it works for you.

Synchronizing PDF files with Latex documents

I am currently in the process of writing up my PhD thesis using Latex. I just realized how spoiled Mac users are when it comes to TeX tool support: nearly all Tex editors support viewing PDF files with source synchronization (iTeXMac2, TextMate, TeXniscope, TeXShop, PDFViewer). On the windows platform, until recently, not a single PDF viewer was capable of doing that including Acrobat, PDF-XChange Viewer, Foxit reader, and SumatraPDF.

Of course under the Windows platform we have the Yap DVI viewer that comes with the MikTeX distribution. Ah, Yap,... This crappy piece of software that becomes less and less usable as the version increases: poor responsiveness when browsing the document, no pre-caching of the images, polling the .dvi file thousands times a millisecond to check whether its content has changed instead of using the proper Windows API, blocking of the .dvi file preventing latex from compiling, and last but not least: loads of modal error messages forcing you to switch to the mouse to get rid of them when you just want to be typing your latex document.

A quick browsing journey on the web suggested me that I was not the only person to feel that frustrated about the current TeX viewing facilities on Windows. But since no one seemed to be interested in implementing the desired features, I decided to give it a try.

So here we are: I have proposed several patches for SumatraPDF that were accepted by the author: the first one implements autorefresh: when you recompile your tex document to pdf, the viewer automatically refreshes itself without any need for user interaction.

The second feature is inverse-search synchronization based on pdfsync: when you double-click on the pdf file it brings you directly to the .tex source file corresponding to that location in the pdf. By default the WinEdt editor is invoked. This can be configured (Menu 'File\set inverse search command line'). For instance the string to start TeXnicCenter is

"C:\Program Files\TeXnicCenter\TEXCNTR.EXE" /ddecmd "[goto('%f', '%l')]"

For WinEdt it is:

"C:\Program Files\WinEdt Team\WinEdt\winedt.exe" "[Open(|%f|);SelPar(%l,8)]"

I have also updated LatexDaemon to make it compatible with pdfsync.

If you want to give it a try, you can download the binary file here.
The changes to the source code have been submitted to the SVN repository at Google Code.

See also these forum posts

My experience with the XML/Docbook/XSLT toolchain

Recently I have tried to learn how to use the Docbook toolchain to produce documentation. My overall impression is not so good. I have many complaints but I am mainly concerned with its poor performance.

Take the editors: I have tried many of them including (XMLMind and Oxygen). On my machine (Athlon XP 1.25GHz, 1.5GB RAM, XP SP2), they all seem to struggle to analyze the XML document as I am typing it. With Oxygen, for instance, there is a noticeable delay of up to 5s when I type which makes it almost useless.
Of course you don't have to use an XML editor as you can edit your document with Notepad. However as the XSLT tools are very strict on the syntax (not like the way browsers interpret HTML), it can be quite hard to edit your document without assistance. Moreover since XML is by nature very verbose, you may end-up spending more time opening and closing corner brackets that actually typing your documents.

Now concerning the XSLT parser, the results speak for themselves: Processing 36.7KB of XML into a 33.3KB HTML file took 34.172s with the xsltproc parser and 38.719s with saxon.net 9.0.0.2. For such a simple file (it's a very small document, not even a book) that's just ridiculous! To give an element of comparison, on the same machine, to compile the TeX book (by Don Knuth) from a 1.35MB file TeX file into a 500-page PDF file weighting 2.23MB it just takes 7.25s!!! Conclusion, if Knuth had written his book using Docbook back in the 70s, he would probably still be waiting for the compilation to finish...

Note:As most of these tools are written in Java (IDE and XSLT included) I guess that part of the poor performance has to be imputed to Java.

Cracklock 3.8.27 released!

A new version of Cracklock correcting some bugs is available here.