Both sets of problems had the same cause, which I only found once I turned error_reporting up to E_ALL:  iconv() or related iconv extension functions failing to return any output, and throwing a Notice-level error usually saying something helpful like “Notice: iconv(): Wrong charset, conversion from `’ to `UTF-8′ is not allowed” where UTF-8 is the charset used by the Pdf or Lucene module.

Normally these would be caught during development, where our environments are configured to display all errors, even Notices.  But this only happens on my production servers so it was not caught until functionality started failing in production.  The moral of that sub-story:  something about Solaris maybe, I’m not sure if there even is a moral there.

At any rate, on the problematic Solaris servers, calling iconv() with an empty first parameter $in_charset causes the above Notice, and the function returns False.  The parameter is empty because I did not specify a value for the optional charset parameter to methods like Zend_Pdf_Page::drawText() and Zend_Search_Lucene_Field::Text().

Personally I think it’s a bit of lazy-ish coding in ZendFramework which allows that parameter to be empty in the first place; behavior for iconv() is not defined in the documentation for an empty input charset.  But I can’t fault them in the end since this behavior seems to be so highly isolated and isn’t even replicatable among our development and QA servers.  So far my only remedy has been to find every single place that an iconv function is used in the ZF codebase, and make sure that the variable for the input encoding, if empty, gets replaced with the value of the iconv.input_encoding config variable.

We’re currently using the latest “gold” version of ZF as of about 10/1/2010, and I have patched the source to eliminate the problem from both the Pdf and Search modules.  If you would like a cvs patch or gnu patch to apply those changes feel free to contact me via email.

Apache’s custom log format shows how to log variables from the Apache environment.  If there’s an environment variable called FOOBAR, you log it with the format string %{FOOBAR}e.  This is great because PHP can inject data into the Apache environment, meaning we can get more or less any data we like into an Apache custom log.

The first Apache configuration step was to create a custom log format that we can then apply to a logfile.  We create the log format in httpd.conf.  Find the log_config_module section and add a LogFormat directive there:

<IfModule log_config_module>
LogFormat "%t %U %D %a %{MY_UID}e" mystats
</IfModule>

In this example, we add a log format named “mystats” which logs these datapoints:  date/time the request was received, URL path requested, time taken to serve the request, remote client IP address, and the environment variable MY_UID.  For our purposes, we are logging these data because our aim is to keep a 7-day rolling log of webserver performance – how long it took to process each request – to monitor our servers more effectively.

Then we set up a custom log for our site’s vhost in httpd-vhosts.conf:

<VirtualHhost 192.168.100.1:80>
...
CustomLog logs/mysite_custom_log mystats
...
</VirtualHost>

This makes Apache start logging every request to that host, in the format defined by our LogFormat directive named “mystats”.

The first step in PHP code, then, is to simply inject the MY_UID environment variable containing the user ID of whoever is logged in.  In our master controller there’s a check to see if the client has an authenticated session, and if they do we just add a quick call to set the environment var:

apache_setenv('MY_UID', $the_current_user_id);

Once this in all in place, you should be able to tail the logfile mysite_custom_log and see that requests are being logged with the requesting client’s user ID, or “-” if no user is authenticated.

We rotate these logs daily.  Our final step is a cron’d PHP script that runs in the wee hours, taking yesterday’s log file, iterating each line, inserting the data into a table… and we’re done!  We logged each request with how long our server processed it, and who was signed in for that request, and we did it without adding any overhead to our application.

So you are hypothetically deploying your ant-built tar’d up software release. Let’s further say for the sake of argument that it contains some business-critical scripts that must keep their executable mode to be run from cron. Oops, those files are no longer executable! You will soon be disappointed at the failure of your cron job to do anything at all, and you may experience some increased email volume accompanied by mild irritation.  If this condition persists, call your doctor, or go back and RTFM again because as the docs imply, this shortcoming can be easily overcome.

Your job is to identify all the files that need to keep their mode, username, or groupname, and specify said property in a special fileset called tarfileset.  In my ant build, let’s say the build root is ${builddir} and contains 2 components called src and lib.  I’ve got one specific file src/foo/exec.sh, and a whole tree of stuff at lib/bar, that have to be executable.  I will need to call the Tar task with two tarfilesets: one to include the executables and set their mode, and a second to include everything else while excluding the executables.

Before:

<tar destfile="${buildfile.name}"
     basedir="${builddir}"
     longfile="gnu"
     includes="src/**, lib/**"/>

After:

<tar destfile="${buildfile.name}" longfile="gnu">
     <tarfileset dir="${builddir}" filemode="755">
         <include name="src/foo/exec.sh"/>
         <include name="lib/bar/**"/>
     </tarfileset>
     <tarfileset dir="${builddir}">
         <include name="src/**"/>
         <include name="lib/**"/>
         <exclude name="src/foo/exec.sh"/>
         <exclude name="lib/bar/**"/>
     </tarfileset>
</tar>

Works for me!

• PageSpot
• Media2Layout

So far today I have compulsively checked the download statistics and somehow 60+ people downloaded PageSpot within about 6 hours of it being published.  I know there are a lot of WordPress blogs and users out there but there must be even more than I assumed.