ezSQL updates
Well, got github working great. Did an update today, including:
- $allowed_columns parameter for quick_insert() and quick_update(). So you can pass in $_POST or $_GET, as well as a list of valid indices to use
- (optional) Exception handling with try/catch (more info)
- Set return type (object, associative array, numeric array) globally through $default_type
Also, I’m updating the wiki slowly, transferring all the useful information onto there, since it’s easy to keep everything together.
Thanks!
ezSQL on github
Learning how to use github, so I created a repo for ezSQL.
http://github.com/nshahzad/ezdb
I plan on creating another version (a fork I guess?) for PHP 5.3 when it hopefully launches next week. It’ll have proper namespace support. Haven’t gotten to memcache just yet (hey, been busy! 
, but at least there’s an easier route of distribution now.
Backups are important!
A huge flight sim site was hacked and destroyed this weekend – avsim.com. An important lesson on why off-site backups are critical! They had two servers, and had a backup of A on B, and B on A. Both were taken out.
Ouch. A shame to see it go. I’ve been going there for 6/7 years now. Good luck to them in getting it back up… they had one of the largest file libraries around.
Been a long time…
Sheesh! Things have been real busy, haven’t had much time to sit down and come up with a coherent post. My next update to phpVMS keeps getting pushed back; it’s been crunch time at work.
Though I’ve been messing with a few things – I cut memory usage by about 50% in phpVMS by using __autoload(). Perhaps that can be a good next post.
But I guess the big news (maybe), is that I’m discontinuing Codon. Not that it affects anybody really except my private projects and the few projects it was used on, but I’d rather concentrate my efforts on my new love – CodeIgniter. I’m really digging it, it’s almost as it’s an exact copy of Codon, with how the models work, etc. Though there are alot of PHP4-isms, I think it’s better for me to focus some efforts on learning and modding that. Codon was a great learning project for MVC (which is what it was always meant as). I don’t think there’s any better way to learn how something works than to build it by scratch. You really get to see the ins-and-outs, the pros and cons and the dilemmas faced by doing something like that.
I’ll be porting my date/time math module over to CI soon, but also some mods I’ve been working on to introduce autoload() into CI (drastic memory improvements), and also maybe implement ezSQL into it, since ezSQL is much more lightweight than their ActiveRecord class. I’m using CI now as part of two broader projects involving phpVMS; right now it’s being used as the code behind the remote API which all the phpVMS installs do talk to. I definately like to squeeze any bit of optimizations I can find.
PHP Benchmarking
I came across a few sites which have some valuable information on the “cost” of certain functions or practices in PHP. It comes in handy to keep in mind and avoid doing, and also to remember that there are better ways of doing things. If your application is coming down to relying on micro-optimization, your application itself needs to be looked at. But there’s no harm in knowing what methods take more time than others. Or of course, you can use ‘em to impress a prospective employer 
- The PHP Benchmark – http://www.phpbench.com/
- lixlpixel PHP Benchmark – http://lixlpixel.org/php-benchmarks/
- Comparing all different functions and uses – http://net-beta.net/ubench/
- PHP Benchmarking Suite – http://phplens.com/benchmark_suite/
There are also some lectures and talks on optimizing:
- Speeding up PHP applications by Derick Rethans
- Simple is Hard by Rasmus Lerdorf (the creator of PHP)
- Performance by Rasmus Lerdorf
- PHP Tips & Tricks by Rasmus Lerdorf
- PHP best practices – The dos and don’ts by Tobias Schlitt & Kore Nordmann
Some good stuff there to look over and to keep in mind, especially obvious things to avoid, such as including the count() inside every for() loop. Actually any of the stuff on http://talks.php.net are a good read, and good to see where PHP has evolved from and where it’s evolving to, and to understand how to leverage the language.
To benchmark your scripts, you can use the PEAR Benchmark package; I have PEAR in my path, so for me it was as simple as including the same code that was in the examples.
Pretty PHP URLS without mod_rewrite
Not everyone has access to mod_rewrite, or doing rewrite rules through nginx or lighttpd, but in PHP, you can do this pretty simply. I’ll go through making this mess:
index.php?module=amodule&page=tasklist&id=10
To:
index.php/amodule/tasklist/10
I’ll be going through the code from Codon, specifically the rewrite module, which can handle this. So a few requirements:
- The first argument is the controller we goto (the module in Codon’s case)
- Arguments can vary based on the controller
- Have a default ruleset if we don’t specify for a module
- The ability to “dynamically” change the arguments on the fly, to accommodate different actions
- Also parse the traditional query string rules
- We can specify types for each argument, as a security measure.
- Use a static class, and have the parameters available through $_GET, as well as from the class
PHP stores the query string in the $_SERVER['REQUEST_URI'] variable as:
[php]
echo $_SERVER['REQUEST_URI'];
# Will output:
/index.php/testing/our/parameters
[/php]
Being relative to the base URL. We’ll work on parsing this later. First we need to build our list of “rewrite rules”. What we’re going to do is store the rules in an associative array, based on our module, and each key has an array, with the rules in order, and we specify the types in that. What’d he say? This will explain it best:
[code]
Array
(
[default] => Array
(
[action] => string
[id] => int
)
[users] => Array
(
[action] => string
[username] => string
)
)
[/code]
So if we look at default:
[code]
index.php/{action}/{id}
[/code]
Where action and ID will be treated as strings. So we’ll build a function for that (Note: This is all in a static class)
[php]
public static function AddRule($module, $params)
{
$set_params=array();
$module = strtolower($module);
# Format the rules, make sure we arrange
foreach($params as $key=>$value)
{
# If it wasn’t done as $key=>$value, just $key,
# set the default value type as a string
if(is_numeric($key))
$set_params[$value]=’string’;
else
$set_params[$key]=$value;
}
self::$rewrite_rules[$module] = $set_params;
# This is for if we’ve already processed the rules
# once. This will allow the rules to be changed
# "on the fly", for example, inside a controller
if(self::$run == true)
{
# Reprocess the rules
self::ProcessModuleRewrite($module);
}
}
[/php]
There is the self::$run “clause”, which sees whether the rules have been parsed or not, and if so, then we’ll just process that particular module. This is to then if the rules are changed on the fly, after we reprocess, then the entire procedure doesn’t have to be done. This will make more sense later.
So we go through, and set the proper types for the parameters.
We call this as:
[php]
CodonRewrite::AddRule(‘default’, array(‘action’=>’string’, ‘id’=>’int’));
# If we do it as keys, it’ll be strings as default, as we’ll see later
CodonRewrite::AddRule(‘users’, array(‘action’, ‘username’));
[/php]
Now we can actually parse the URL, which is the bit we’re interested in. I use explode()’s to parse out the pieces, as I find that to be the most legible:
[php]
public static function ProcessRewrite()
{
$URL = $_SERVER['REQUEST_URI'];
# Get everything after the .php/ and before the ?
$params = explode(‘.php/’, $URL);
$preg_match = $params[1];
$params = explode(‘?’, $preg_match);
$split_parameters = $params[0];
# Now check if there’s anything there (we didn’t just have
# index.php?query_string=…
# If that’s all, then we grab a configuration setting that
# specifies the default rewrite, ie: news/showall
# Which would eq. passing index.php/news/showall
if($split_parameters == ”)
{
$split_parameters = Config::Get(‘DEFAULT_MODULE’);
}
# Now we split it all out, and store the peices
self::$peices = explode(‘/’, $split_parameters);
$module_name = strtolower(self::$peices[0]);
if($module_name == ”) # If it’s blank, check $_GET
{
$module_name = $_GET['module'];
}
self::$current_module = $module_name;
$_GET['module'] = $module_name;
# Create the object to hold all of our stuff
self::$get = new stdClass;
# If we haven’t specified specific rules for a module,
# Then we use the rules we made for "default"
if(!array_key_exists($module_name, self::$rewrite_rules))
{
$module_name = ‘default’;
}
# This parses now the rules for a specific module
self::ProcessModuleRewrite($module_name);
# And this tacks on our $_GET rules
parse_str($_SERVER['QUERY_STRING'], $get_extra);
$_GET = array_merge($_GET, $get_extra);
# Add the $_GET to our object
foreach($_GET as $key=>$value)
{
self::$get->$key = $value;
}
self::$run = true;
}
[/php]
Some of it is pretty straight forward, but that’s half of the “workhorse”, next is the ProcessModuleRewrite() function:
[php]
public static function ProcessModuleRewrite($module_name)
{
# Make sure it’s valid
if(is_array(self::$rewrite_rules[$module_name]))
{
# Walk through every peice of the array, $key is the
# index name, and $type is well, the type
$i=1;
foreach(self::$rewrite_rules[$module_name] as $key=>$type)
{
# Each peice, which was saved above as the exploded URL
$val = self::$peices[$i++];
# Convert to type specified
if($type == ‘int’)
$val = intval($val);
elseif($type == ‘float’)
$val = floatval($val);
# We can do any other processing we want here
# Add it both into the $_GET array, and into
# our object
self::$get->$key = $val;
$_GET[$key] = $val;
}
}
}
[/php]
So as we see, this is where the types come into play, and we do our conversions. We can also strip any slashes here, and whatever other stuff. So now we can do:
[php]
CodonRewrite::AddRule("default", array(‘action’=>’string’, ‘id’=>’int’));
CodonRewrite::AddRule("users", array(‘action’, ‘username’));
CodonRewrite::ProcessRewrite();
echo CodonRewrite::$get->module;
[/php]
What I do is for a controller, I’ll assign $this->get to CodonRewrite::$get, so inside a controller, I can do:
[php]
class MyModule extends CodonModule
{
…
function Controller()
{
if($this->get->action == …)
…
}
}
[/php]
And since rules can be “redone” on the fly:
[php]
class MyModule extends CodonModule
{
…
function __construct()
{
# Since I see it’s a save, I’m modifying the rules a bit here
# so I can include another extra parameter in
if($this->get->action == ‘save’)
{
CodonRewrite::AddRule(‘mymodule’, array(‘page’, ‘action’, ‘id’=>’int’, ‘content’));
}
…
}
function Controller()
{
if($this->get->action == ‘save’)
…
}
}
[/php]
I call this the Rewrite functions as the first thing in my main controller, and then I can base which module/controller to call based on these rules here. Make’s it simple for parsing, and pretty easy to see how the rules are made.
I hope that helps someone!
ezSQL (more stuff)
I think I’m going to setup a SVN repository for ezSQL, since I’ve got a few ideas. I’m going to integrate memcache as one of the caching options, in addition to the flat-file caching it does now. Any other suggestions?
Nginx + PHP Query Strings
Nginx (Engine-X) is the sweetest web server I’ve used. It’s running my VPS now, CPU usage has barely budged, even with a decent number of visitors. I ran into a problem, which I was searching for a solution before I cracked at it myself. The solution ended up being incredibly simple, but still may help someone else who’s searchin’ for it.
With nginx, PHP is passed of to FastCGI PHP process to parse and execute. It relies on a location rule to find out what are PHP files. But it would fall apart using URLs like:
index.php/some/query/string/parameters
Apache doesn’t have any trouble, with it, but nginx, if you looked at the error log, it’ll be about the directory not existing. (Hmm, parsing that in PHP sounds like a good idea for another post… noted).
So if we check out the current rewrite rules:
[code]
location ~ \.php$ {
include /etc/nginx/conf/fastcgi_params;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /path/to/public/$fastcgi_script_name;
}
[/code]
The fix is pretty simple:
[code]
location ~ \.php<strong>(.*)</strong>$ {
include /etc/nginx/conf/fastcgi_params;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /path/to/public/$fastcgi_script_name;
}
[/code]
Adding in the (.*) after the .php allows the PHP files to be picked up, and those additional query string parameters will get passed along into FastCGI.
Simple fix!
Shell scripts no more!
This weekend, I started my server migration, over to Slicehost. It went well, now I’m running on a lean ‘n mean nginx install. As I was moving my Subversion repositories, I was dreading having to move all my shell scripts, which I used to build and deploy some of my applications (outlined in this post). I was thinking there had to be a better way, after all, Ruby has Capistrano, and though it can be used with PHP, I didn’t want to have to install Ruby, etc etc. After some searching (not much), I found Phing, which looked like exactly what I needed. Sweet!
Phing takes an XML file, which you can define all the transformations and instructions. It has sets called “targets”, which are different stages of the build and transformation. They have a basic example here, but I’ll go through my phpVMS build.xml, since I do a number of things they don’t show in the examples, like checking out from Subversion, some variable replacements, and building tars for different “stages”. Having targets was great, since you can select which targets to run, so now I can combine a “beta” and “release” builder, instead of having separate files for it (as I did with the shell scripts). My goals are the same as in the SSH scripts:
- Automatically build beta, full, and update versions from SVN
- Update revision numbers within several files using tokens or regex
- Copy it to the test site
- Automatically generate PHPDoc API documentation
Phing is perfect for this. I had sets of painful-to-maintain and update SSH scripts to do this same thing. And it has a simple PEAR installer. I installed it using:
[bash]
sudo pear channel-discover pear.phing.info
sudo pear install phing/phing
[/bash]
And that was it! Installed. They do have instructions for a I now had “phing” available on the command line:
[bash]
phing -version
Phing version 2.3.3
[/bash]
Now we can get started, building our XML file. The file starts with:
[xml]
<xml version="1.0">
<project basedir="." default="beta" name="phpvms">
</project>
[/xml]
That’s our basic container; as you can see I set the project name, the default “target” or stage to run, and the base directory. Everything we do/add will go inside the project tags.
Now, I’m going to add some common “properties”, or variables, that will be used throughout the script, to make it easier, and not have to type out paths over and over:
[xml]
<!– Build properties –>
<property name="project.wcdir" value="/path/to/phpvms.wc" />
<!– This is the path to where a clean export can be done of SVN –>
<property name="project.exportdir" value="/path/to/phpvms.export" />
<!– This is the path where the build binaries can go –>
<property name="dest.dir" value="/path/to/public/files/" />
<!– This is the path where the "test build" goes (phpvms.net/test) –>
<property name="test.dir" value="/path/to/public/test/" />
<!– This is where the docs generated by phpdoc will go –>
<property name="php.doc" value="/path/to/public/docs/api/" />
<!– URL to our SVN repository –>
<property name="repository.url" value="svn://phpvms.net/phpvms/trunk" />
<!– Username and password, blank since the above can be accessed anon –>
<property name="repository.user" value="" />
<property name="repository.pass" value="" />
[/xml]
Use absolute paths if you can, otherwise, it’s relative to the basedir property defined in the project tag.
The reason for having a working copy and an export directory for SVN is this – the checkout directory will be used to determine the latest revision, since the svnlastrevision command requires the path to a working copy. The export directory is to hold a clean export; without all the subversion specific data that’s included in a checkout. We’ll package that up, and use that to copy to a test site.
Next, I define the targets, or stages, I’m going to use: prepare, build, subs, phpdoc, beta, and release. They are setup this way in roughly the order they will run. The order doesn’t matter, since we determine the order of the targets by using the “depends” property. I’ll explain each one in detail:
[xml]
<target name="build" depends="prepare">
</target>
<!– This is where we will do our variable substitutions
It depends on the target above, primarily in our case,
replace all version numbers –>
<target name="subs" depends="build">
</target>
<!– This is the PHP doc target, it’s its own target
since we will call it from beta and release –>
<target name="phpdoc">
</target>
<!– In this target, we will create a "beta" release.
It depends on the subs build. We will call this
target, or the release target specifically from the
command line when we run phing –>
<target name="beta" depends="subs">
</target>
<!– In this target, we create a "release", and it depends on
the "subs" target above. We’ll also specifically call this
from the command line –>
<target name="release" depends="subs">
</target>
[/xml]
So now that we have an outline, we can go through each target, doing what we need to do, first our “prepare” stage:
[xml]
<target name="prepare">
<!– First we are going to delete the export directory
So we have a clean copy that we export from SVN.
Then we re-create it using the mkdir command –>
<delete dir="${project.exportdir}" failonerror="true" verbose="false" includeemptydirs="true" />
<mkdir dir="${project.exportdir}" />
<!– Now we create the working-copy directory,
where the checkout will be –>
<mkdir dir="${project.wcdir}" />
</target>
[/xml]
We’re using our properties which we defined, as ${propertyname}. Don’t need to do any escaping, etc, just use ‘em right in the strings.
Next is our build target stage, which will do the Subversion operations of getting the latest revision.
[xml]
<target name="build" depends="prepare">
<!– We are checking it out to the checkout directory –>
<svncheckout todir="${project.wcdir}" repositoryurl="${repository.url}" nocache="true" force="true" password="${repository.pass}" username="${repository.user}" svnpath="/usr/bin/svn" />
<!– We’re now going to get the last revision
from the above checkout. The last revision
number will be stored in the propety called
svn.lastrevision, which we can access by
${svn.lastrevision}. No more parsing results
from the command line… wahoo! –>
<svnlastrevision svnpath="/usr/bin/svn" propertyname="svn.lastrevision" workingcopy="${project.wcdir}" />
<!– Export a clean copy which we can package up –>
<svnexport todir="${project.exportdir}" repositoryurl="${repository.url}" nocache="true" force="true" password="${repository.pass}" username="${repository.user}" svnpath="/usr/bin/svn" />
</target>
[/xml]
It’s great how Subversion operations can easily be done. Next target is subs, to replace any tokens and variables we define in the files.
[xml]
<target name="subs" depends="build">
<!– Reflexive allows work on a set of files, and
a "reflexive" block is required for a filterchain–>
<reflexive>
<!– Here we define the files that we’re going to
work in using a filterchain –>
<fileset dir="${project.exportdir}">
<include name="changelog.htm" />
<include name="install/install.php" />
<include name="install/update.php" />
<include name="install/install.sql" />
</fileset>
<!– Setup a filterchain now, the above files
pass through these filters –>
<filterchain>
<!– Instead of doing a regular expression match,
Phing supports token replacement. I use the
##REVISION## token wherever I want it to be –>
<replacetokens endtoken="##" begintoken="##">
<!– Replace the REVISION token with our svn.lastrevision property –>
<token value="${svn.lastrevision}" key="REVISION" />
</replacetokens>
</filterchain>
</reflexive>
</target>
[/xml]
Next is our phpDoc function, which is sort of standalone, in the fact that there are no depends for it.
[xml]
<target name="phpdoc">
<!– Setup the basic options –>
<phpdoc title="API Documentation" output="HTML:Smarty:PHP" sourcecode="no" destdir="${php.doc}">
<!– These are the files that are going to be
included as part of the documentation –>
<fileset dir="${project.exportdir}">
<include name="core/classes/*.php" />
<include name="core/common/*.php" />
<include name="core/modules/**/.php" />
</fileset>
<!– This lists the files which are part of the
official documentation –>
<projdocfileset dir="${project.exportdir}">
<include name="changelog.htm" />
</projdocfileset>
</phpdoc>
</target>
[/xml]
Now that we’ve got the basics down, the next two targets are “beta” and “release”. Each of these handles a different set of tasks, though they are basically the same. First for beta:
[xml]
<target name="beta" depends="subs">
<echo msg="Creating archive…" />
<!– Delete the files first, since they are not updated,
The tar function will just append onto them –>
<delete file="${dest.dir}/phpvms.beta.tar.gz" />
<delete file="${dest.dir}/phpvms.beta.zip" />
<!– Create the tar file, specifying the files to include –>
<tar compression="gzip" destfile="${dest.dir}/phpvms.beta.tar.gz">
<fileset dir="${project.exportdir}">
<include name="*" />
</fileset>
</tar>
<!– I was having trouble with the built-in ZIP functionality
So I resorted to using the command line. Either way, this
shows off the command line functionality that Phing has –>
<exec escape="false" command="cd ${project.exportdir}; zip -D -r ${dest.dir}phpvms.beta.zip ." />
<echo msg="Copying to test site" />
<!– Copy it to the test site, so there’s an updated
copy available there to debug–>
<copy todir="${test.dir}" overwrite="true">
<fileset dir="${project.exportdir}">
<include name="*" />
</fileset>
</copy>
<!– Call the phpDoc target that we created –>
<echo msg="Creating phpDoc" />
<phingcall target="phpdoc" />
<echo msg="Files copied and compressed in build directory OK!" />
</target>
[/xml]
And now for the release target. Release a bit different than beta – it creates two different archives – one full version, and one update. The difference essentially is that the update copy has the local configuration file removed, where database settings are stored, and other settings which are kept locally.
[xml]
<target name="release" depends="subs">
<echo msg="Creating release builds…" />
<!– Delete all the files, start fresh –>
<delete file="${dest.dir}/phpvms.full.tar.gz" />
<delete file="${dest.dir}/phpvms.full.zip" />
<delete file="${dest.dir}/phpvms.update.tar.gz" />
<delete file="${dest.dir}/phpvms.update.zip" />
<!– Create the tar file –>
<tar compression="gzip" destfile="${dest.dir}/phpvms.full.tar.gz">
<fileset dir="${project.exportdir}">
<include name="*" />
</fileset>
</tar>
<!– Create the zip file –>
<exec escape="false" command="cd ${project.exportdir}; zip -D -r ${dest.dir}phpvms.full.zip ." />
<!– Now create the update build, which basically deleting the
local.config.php file –>
<echo msg="Full release created, creating update" />
<!– Delete the local.config.php file, since that’s not distributed
with an update–>
<delete file="${project.exportdir}/core/local.config.php" />
<!– Create the tar file –>
<tar compression="gzip" destfile="${dest.dir}/phpvms.update.tar.gz">
<fileset dir="${project.exportdir}">
<include name="*" />
</fileset>
</tar>
<!– Create the zip file, using command line instead
I kept getting out of memory errors using the built-in
ZIP functionality. –>
<exec escape="false" command="cd ${project.exportdir}; zip -D -r ${dest.dir}phpvms.update.zip ." />
<!– Call the phpDoc target –>
<echo msg="Creating phpDoc" />
<phingcall target="phpdoc" />
<echo msg="Files copied and files moved, done!" />
</target>
[/xml]
And that’s it! Now we can run it using:
[bash]
phing -buildfile /path/to/build.xml beta
phing -buildfile /path/to/build.xml release
[/bash]
Using the appropriate one we want to use. In my SVN post-commit file, I run the beta line, in order to have that build every time.
I attached a fully doc’d version here (same as the above, just in one file). I hope it helps!
ezSQL Database Library (Improved!)
Note: I’ve made a dedicated page for ezSQL-related items, you can view it here
I’ve been using ezSQL for a long time, as my “database driver”. It’s an awesome little class, that you can easily use to get database results, and return in different formats. I’ve made a bunch of changes and updates to it, so I thought I’d put them out there and share them.
Along the way, I’ve made some updates and changes to it, to suite my requirements, including format PHP5 support. I’ve updated the error handling/logging, so now you can call:
[php]
$sql->error(); // Get the error string
$sql->errno(); // Get the error number
[/php]
Respectively after queries to return the “real” status of a query. the debug() function has been added to, so at call time, you can pass to bool whether to display the result on the screen, or pass it back as a string
I’ve also added several utility functions:
[php]
$sql->quick_select();
$sql->quick_update();
$sql->quick_insert();
// quick_select() example:
$columns = array(‘column1′, ‘column2′);
$sql->quick_select(‘table_name’, $columns, ‘LIMIT 10′);
[/php]
To make it easier to do simple SELECT’s and UPDATEs, and INSERTs.
Another thing it was missing with MySQLi support, so that has been added (though I have not had the time to add support for statements (though that can be accessed rather easily). I had added MSSQL support, but the file has gone AWOL (Icreated and used it for a specific project a while ago). I will update when I get a hold of it.
And another thing I added was a static interface class, which I use all the time on PHP5 projects:
[php]
DB::init(‘mysql’);
DB::connect(‘username’, ‘password’, ‘database_name’, ‘localhost’);
// Now anywhere in your script:
$row = DB::get_row(‘…’);
// Or
$results = DB::get_results(‘…’, ARRAY_A);
[/php]
Makes it much easier to access the database function, without having to do $db = DB::getInstance(), in every single function to get a singleton object of the database. Since that’s for PHP5 only, the whole library has been updated for PHP 5 support, with public/private functions and constructors/destructors as well.
I also condensed it to just one include() now, just keep all the files in the same place (include ‘…/DB.class.php’). From that you can either use the static class, or declare a new ezSQL object.
I also added in phpDoc blocks on all the functions, since IntelliSense is awesome and really helpful.
I hope these improvements make it easier for everyone. Of course, the original credit goes to Justin Vincent. There are some docs and examples here as well.
