PHP is not Java! - PHP

Dumping MemcacheD Content (Keys) with PHP

nospam@example.com (Gaylord Aulke) — Thu, 28 Aug 2008 20:24:48 +0200

MemcacheD is a very cool piece of software. When i did some optimization of a cluster based webapp lately, i was wondering how Memcache was speading my cache entries over the cluster. So i did some research for monitoring tools. A simple approach to monitoring would be to use Cacti to monitor status values like Cache usage, hits/sec etc. This can be done by a template like this. An alternative was provided by Harun Yayli: His memcache.php which somewhat resembles the APC status page and now is a part of PECL/memcache is easy to setup and works well. It also allows to dive into the data structures but you need to decide for one server first, then click into a "Slab" and then you see the keys. Inspired by this, i wrote a small script that fetches all data from a memcache cluster, gets all keys out of it and then sorts and displays them in a list. Yes it it ugly and yes: the memcacheD is not answering other requests while doing a cachedump. But i found it very useful. Maybe somebody has something more advanced that he wants to point me to? This could save a lot of time...

But here is my naive approach: Assumed you have a memcache object in PHP 5 with some servers registered using addServer Method, the following can be used:

	    $list = array();
	    $allSlabs = $memcache->getExtendedStats('slabs');
	    $items = $memcache->getExtendedStats('items');
	    foreach($allSlabs as $server => $slabs) {
    	    foreach($slabs AS $slabId => $slabMeta) {
    	        $cdump = $memcache->getExtendedStats('cachedump',(int)$slabId);
    	        foreach($cdump AS $server => $entries) {
    	            if($entries) {
        	            foreach($entries AS $eName => $eData) {
        	                $list[$eName] = array(
        	                     'key' => $eName,
        	                     'server' => $server,
        	                     'slabId' => $slabId,
        	                     'detail' => $eData,
        	                     'age' => $items[$server]['items'][$slabId]['age'],
        	                     );
        	            }
    	            }
    	        }
    	    }
	    }
	    ksort($list);

I then added a very simple HTML Table output to list the result array and found out some interesting details about my application that i dont want to share here

It worked for me but it could be that for bigger Memcache Clusters (mine has only 10 servers) or big content tables this will not return the complete content. More to come...

Notification Free PHP Coding

nospam@example.com (Gaylord Aulke) — Sat, 21 Jun 2008 16:22:13 +0200

Sorry guys, flamebait again. But i need to say this: I HATE NOTIFICATION FREE PROGRAMMING in PHP!
If you want to code Java, go ahead! If you use PHP, enjoy the simplicity it brings!
Today i found this code by someone who really wants to do it right:

   $email = (key_exists('email', $values)) ? $values['email'] : null;
   $password = (key_exists('password', $values)) ? $values['password'] : null;
   $firstName = (key_exists('firstname', $values)) ? $values['firstname'] : null;
   $secondName = (key_exists('secondname', $values)) ? $values['secondname'] : null;

Honestly: This does not make any sense at all. It just pollutes your code with technical constructs that don't contribute to a solution.

UPDATE: Just to make this clear: I am _not_ talking about not input filtering here! Of course every input needs to be whitelist-filtered. And every ouput escaped to prevent XSS. What i mean is internals of you script: You can count on PHP to do things for you. This is exact because PHP does it in exact the same way everytime the script is executed.

PHP vs. Java

nospam@example.com (Gaylord Aulke) — Sun, 18 May 2008 13:01:23 +0200

Everyone working with PHP is probably asked quite often what the difference is between PHP and Java (or C#/.Net). Besides the usual aspects: scripting vs. compiled and in-process vs. seperate process and Multithreading etc., i think there are some "soft facts" that might be even more important for commercial software development.

For example: In Java (or C#) you can code any architecture for any Problem. Some will need more development time and more hardware, others less. In PHP you still have a lot of ways to solve a problem but not as many as in other languages because of the limits to object lifetime. Some of the possible solutions will lead to a dead-end road in terms of performance and maintainability. Therefore, intelligent PHP developers tend to communicate first. They search for proven solutions, maybe existing PHP-Extensions and working code before defining an architecture (and i dont mean Design Patterns, but examples of working solutions). Maybe it even helps in this case that quite some PHP developers don't have a solid computer science background and therefore they need this kind of inspiration to find a solution at all. On the other side, people tend to be proud if they have something working in PHP thus blog about it and opensource their architecture and code.

Of course you find all this also in Java or C#, but much much more in the PHP area. It seems like the "Not-Invented-Here" syndrome is less mighty here or people are more motivated to share.

Anyway, the result is: Even though there are not so many standards in the PHP world, successful developers have a common understanding about the do's and don'ts in PHP. This unwritten standardization leads to a very interesting fact (also shown in the Plat-Forms contest): PHP solutions of different teams (of comparable skill level) are much more consistent than in Java (or probably C# (Prof. Prechelt, please forgive me this un-scientific deduction)). Meaning: PHP appears more predictable than other programming languages. Now this is a fact that the business decision takers might be interested in...

requirements management, the php way

nospam@example.com (Gaylord Aulke) — Tue, 15 Apr 2008 09:23:59 +0200

In typical PHP projects, people are afraid of spending more hours in development that actually planned. As a main reason for this exceeding the estimated (thus paid) budget of hours, usually "technical problems" are given. But the experience (and some empirical studies) show that this is not really true: Especially for bigger projects (>100 person days) a much bigger danger lies in understanding the requirements right. Of course we work iteratively and of course we re-estimate each feature right before we start implementig it. But our customers have internal processes and thus they need to scope the project very detailed upfront. So we need to tell them what we will deliver and what cost are involved with it. Actually, what we need is an adapted form of planning game as described in agile methods: We need to gather all functional requirements from an unstructured document and we need to roughly estimate on a basis of these requirements in order to define features and scope the project right.

There are a lot of books about classical requirements management for the usual waterfall project model. In the PHP world, things are more dynamic and change more often. We have solved this problem for a quite big project currently. Our PHP-minded approach looks like this:

1. We get a functional requirements document from the customer. This is his "wishlist"

2. We derive the actual requirements from this document. This is important because the customer tends to distribute similar requirements via the whole specification and thus, the structure of his document is usually not suitable for our project organisation.

We list the requirements in an excel sheet that has the following columns:

- Chapter in the original document of the customer
- Brief description of the feature
- Rough first estimation of the development effort (we use person days, but you could also use "feature points")
- Comment field for assumptions regarding the estimation
- a feature number and feature name

While we isolate requirements, we immediately associate them with features. If no feature had been defined before that matches the new requirement, we add a new one. In excel that is easy: just type a new feature name, the completion feature of excel helps to aviod mis-spelling when referring to a previously defined feature. Feature numbers we assign in a separate list and also write to our requirements list.

We put every requirement we find into the list. Even if it is double (i.e. mentioned in a different chapter). This way, we build a relation between chapters in the requirements document and our feature numbers.

3. Now we sort the list by feature. After that we see all the requirements that were redundant. We keep them in the list, but reduce the estimated hours if the redundant ones to 0. This way we keep the relation but avoid doubling the estimated cost.

4. My new favorite feature in Excel: Pivot Tables. I am not a microsoft fan, but this is really cool: Excel can aggregate all estimated hours per feature in such a table. The result is a list if disctinct features with the associated estimated days.

5. This table we can then discuss with the customer. We know where the effort comes from and we know where in the specification document the requirements for this were hidden. The customer might then say: "Let us talk about this feature, i don't want to spend so much time on it". We can then look into the requirements list for this feature and check what the biggest points are and if we can reduce effort here.

6. During this discussion, we add another 2 Columns to our excel sheet: New Estimation and New Comment. After (or better: during) the discussion with the customer, we fill in the newly estimated person days and the comment, what was changed towards the old estimation in these columns. Then we define another Column in the Pivot Table that aggregates this new estimations per Feature for us.

7. Finally we write down the features along with the references to the specification document, the new estimated time and the new comments (if any) in a technical specification document, add some design and architecture information and we have our offer.

We tried quite a lot of approaches in projects of different sizes but this seemed to be the best way so far for us to deal with big specification documents the PHP way. Maybe someday if i have a lot of time, i will write a database driven PHP app replacing the Excel sheet. But so far, it works great this way

Zend Download Server

nospam@example.com (Gaylord Aulke) — Tue, 25 Mar 2008 12:45:30 +0100

The Zend Download Server (ZDS) (Part of the Zend Platform product) takes over long running download processes from the apache/php running the actual web application. The download is then handled by a very lighweight process, saving resources. This is very similar to X-LIGHTTPD-send-file of lighty or its apache-port mod_xsendfile (while this only does half of the job). The advantage of ZDS is that it can also send strings and streams that do not nessecarily reside as files on any harddrive. It is as easy as calling:

zend_send_file(<filename>,<mimetype>,<headers>)

zend_send_buffer($phpVar,<mimetype>,<headers>).

Then, the PHP process is terminated and the content is delivered using the ZDS.

So much the theory. There are three issues to consider though:

ZDS discards all headers that have been set by the PHP header() function in advance. If you want to send headers (like content-disposition of filename), you need to use the optional <headers> parameter.
To determine the mime type of the file to be sent, ZDS uses its own mime-Type-Lookup-File. It does not use the apache mime.types.
If sending a file, make sure that it resides on local file system since ZDS uses mmapping to access the file, which is quite inefficient (if not dangerous) on network mounts like NFS3.