Basic tech stuff

Programming and Linux administration

PHP is seriously sick

Posted by Daniel Brahneborg on 2007 March 21

It’s obvious that ‘true‘ should be considered true and ‘false‘ should be considered false in an if statement.

It’s sane that ‘1‘ should be true, but slightly annoying sometimes that ‘0‘ is false. Especially since '0==null‘ also evaluates to true. What idiot decided that?

Then it gets worse. The expression ‘array()‘ is also false, making it impossible to use “if ($variable)” to decide if the variable has been set to anything.

On top of that, ‘array()==null‘ also evaluates to true. WTF? Seriously?

After some searching, my little if statement now looks like this: “if (is_array($variable)) …”. Gee, thanks.

When coming from an object oriented world, an object is an object, and an explicitly created object is certainly different from null. That’s so fundamental it goes beyond saying.

Combined with the fact that a normal variable assignment always (unless you’re using “&=“) makes a deep copy of an array, my little loop to create a nested array of arrays, became a nightmare. It’s three times as long as the Ruby version, and with lots of duplicated code since I always have to populate each array before adding it into the containing object.

And people actually like this language? Why?

Andra bloggar om: , .

(Edit: Corrected the true/false thing found by Svava)

Posted in php, programming | 12 Comments »

Blowfish decryption in Ruby, revisited

Posted by Daniel Brahneborg on 2007 March 14

I wrote earlier about my struggles to get decryption to work in Ruby. Today I got it working, with good performance.

The solution was to write a Ruby extension in C, calling the Blowfish and AES functions in the OpenSSL library directly. Converting from Ruby types to C types and back is easy, so the process of figuring out how to do it, writing the code and tweaking it until the test suite passed again, took less than a day. Now time is measured in seconds again.

The thing that surprised me the most was when I changed decryption of file names and other short strings, to also use the C version. This gave yet another performance boost, cutting the time for the test suite from 5-6 seconds to less than 1. If nothing else, it shows that Ruby needs to get faster.

Andra bloggar om: , , .

Posted in encryption, programming, ruby | 10 Comments »

Conditional parameters in Ruby

Posted by Daniel Brahneborg on 2007 March 12

All cool Ruby hackers probably use this all the time, but I think it’s worth mentioning anyway.

I have a generic send_operation() function that takes a hash table of parameters to send. It’s used as the backend for the more user friendly things like login() etc. Some of the functions take a single, optional parameter. First I used the simple version:

args = nil
(args ||= {})[key] = value unless value.nil?
result = send_operation(op, args)

However, it can be made much nicer, using the fact that everything has a value. Combined with the “unless” modifier, I got a single line version that looks like this:

result = send_operation(op, ({ key=> value } unless value.nil?))

The parentheses around the “unless” construct is necessary, or Ruby gets a bit upset. Since there is an extensive unit test suite for all functions in all versions, it was easy to verify that it works correctly.


Andra bloggar om , .

Posted in programming, ruby | Leave a Comment »

Blowfish decryption in Ruby

Posted by Daniel Brahneborg on 2007 March 9

For a project that will soon be released as Open Source, I need to decrypt blowfish encrypted data in Ruby. There are several ways, none of which makes me happy.

1. The Crypt library in pure Ruby. The current version has a few bugs, but after adding a couple of patches, it decrypts the data perfectly. Downside? It’s slow. Slow as hell. Data that takes 3 seconds to decode in C takes 300 with this library.

2. The EzCrypto library. It’s a frontend for the OpenSSL libraries, and should therefore be quite a bit faster. I don’t know, because I simply can’t get it to work. I’ve followed all the descriptions, but it never manages to decrypt the data correctly. My code looks something like this, assuming that the IV is in the beginning of the encrypted data:

def decrypt(key, data)
    @driver = EzCrypto::Key.new(key, :algorithm => "bf-cbc")
    @driver.decrypt(data)
end

3. Using the Ruby OpenSSL libraries directly. This doesn’t work either. Here I do like this:

def decrypt(key, data)
    @driver = OpenSSL::Cipher::Cipher.new("BF-CBC")
    @driver.decrypt
    @driver.key = @key
    @driver.iv = data[0..7]
    clear = @driver.update(data[8..-1])
    clear << @driver.final
    clear
end

4. Using a shell escape to the openssl binary, using the flags -K and -iv to set the encryption parameters. Doesn’t work either.

5. Using a shell escape to my own binary, which uses the BF* functions in OpenSSL directly. Works like a charm, even if the 900 fork/exec/wait calls bring up the total time to 13 seconds. And of course, it doesn’t work with other encryption algorithms unless I add those explicitly.

So now I have the choice between five versions that all suck. Or rather two, if I only count the versions that actually work. Life is fun.

Andra bloggar om: , , ,

Posted in encryption, programming | 7 Comments »

Web services are dead, long live web applications

Posted by Daniel Brahneborg on 2007 March 3

Many years ago SOAP was born. To be able to write clients and servers, it got a specification called WSDL. Unfortunately it became too general, allowing for all kinds of call patterns, SMTP transfers, lots of different encodings and styles and whatnot. This made it too big, so there are still no tools that fully support it. JAXP probably comes closest, or maybe Dotnet.

The version that everybody uses is WSDL 1.1. WSDL 1.2 was supposed to be released in 2003. There is still no support for it. Now they’re talking about WSDL 2.0, which is even more useless.

In the real world, people used XMLRPC instead, just as they used Spring and Hibernate instead of the stupid “container based management” which only made things too complex. The latest version of Ruby on Rails gave support for REST, and now it seems that the two worlds are coming together, in something called WADL. It’s very similar to WSDL, but with the useless stuff thrown out, and only for REST style applications. The specification is less than 40 pages, which means it’s actually possible to implement tools for it.

I really, really hope something like this will catch on, because the current situation with web services is a nightmare. Everybody wants it, but for everything that is more complex than a Hello World, it’s useless. If you want to support other languages than Java you have to hand craft every single XML request. Completely insane.

Andra bloggar om: , ,

Posted in programming, rails, wadl, webservice | Leave a Comment »

Lessons to get embedded Perl to work in C

Posted by Daniel Brahneborg on 2007 March 1

The theme for this week has definitely been “embedding”. First Ruby, then Perl. Ruby was easy. Perl, being much more mature, should have been rather easy as well. Not quite.

First, it really didn’t like having my call perl_alloc() in one thread and using it in another. Well, that was easy to fix.

The documentation contain enough information about all the ENTER, SAVETMPS, PUSHMARK etc to get the stack correct. However, the only data that was sent was integers and strings. I wanted to send a couple of hash tables, some of them as elements in other hash tables. From the perlguts man page I found the newHV() and hv_store() functions, which looked promising. For some reason it just never wanted to work properly. Either the entire application crashed like crazy, or the perl function simply never got the data. I was starting to get a bit annoyed at this point.

Suddenly I found a typo! I had sent a normal C struct instead of an HV*. No wonder it crashed! Now it stopped crashing so much, but the perl function still didn’t get anything.

I went back to basics, trying to pass hash tables in hash tables from perl to perl. That turned out to be a complete mess, since the values in a hash table only can be integers or strings. Jay, great language. Well done, boys. After a while I found “references”. To put a hash table as the value of another hash table, one had to write beautiful code like this:

sms{'MESSAGE'} = "hello plugin";
$request{'sm'} = %sms;
func(%request);

To extract it, I had to use more beautiful code (both versions of syntax work, but I like variety).

my $sms = ${$request}{'sm'};
my $mess = $sms->{'MESSAGE'};

In other words, you can’t put hashes in a hash table, but a reference to a hash table is ok. In C, this would look like this:

hv_store(smsObj, "MESSAGE", 7, value, 0);
hv_store(requestObj, "sms", 3, newRV_noinc((SV*)smsObj), 0);

Now data passed through correctly.

There are still problems, of course. First, I haven’t verified that data can flow back from perl to C. Second, I’m not sure I’ve gotten all reference counts right, so it might leak memory. Third, it crashes like crazy when calling perl_destruct(). It might help calling this from the same thread as perl_construct(), of course.

Andra bloggar om: ,

Posted in programming | Leave a Comment »

Hard to find documentation about embedding script languages

Posted by Daniel Brahneborg on 2007 February 27

It was quite easy to find information on how to embed Ruby in a C application. The book “Programming in Ruby”, the ruby.h file and some googling, and it got together quite nicely. At least once I learned to always wrap calls with rb_protect and store everything as globals.

Feeling a bit proud, I started looking for information on how to do the same thing with PHP, Perl, Python and whatnot. For some reason, I could hardly find anything at all. Not even the documentation for Parrot contained enough information.

For PHP, it seems one should use something called SAPI. However, neither php.net nor zend.com contained any information about it. For Python it was slightly better, but not much.

Is it really that unique to want to be able to run scripted code from within an application, being able to pass parameters in, getting values back? Odd. Very odd.

Andra bloggar om: , , ,

Posted in programming | Leave a Comment »

Embedded Ruby was quite easy

Posted by Daniel Brahneborg on 2007 February 26

I spent some time embedding the Ruby interpreter into an application today. It was a bit confusing at first, but once I made sure all method calls were surrounded by rb_protect() and stored all variables as globals to keep the garbage collector from eating them, things progressed quickly.

The fun thing is that the documentation said that it wasn’t written to be easy to embed, but it still looked a lot easier than the infinitely complex mod_perl. That says a lot about Ruby, I think.

Posted in programming | Leave a Comment »

Can you write smarter code?

Posted by Daniel Brahneborg on 2007 February 26

Today the Swedish newspaper Computer Sweden has a job ad with the above headline, and a code snippet:

private int extraRounder(float nr) {
  switch (nr) {
    case Float.parseFloat("4.0"): return 4;
    case Float.parseFloat("4.5"): return 4;
    case Float.parseFloat("5.0"): return 5;
    case Float.parseFloat("5.5"): return 5;
    default: return 0;
  }
}

First, using an unnecessary switch is a bit stupid, yes.

Second, it doesn’t even work, since the comparisons with floats never will match due to rounding errors. Because of this, practically all programmers in the world can write smarter code than this (but see the next paragraph). Therefore, the requirements on the applicants can’t be very high. Not really a company I’d like to work for.

The thing that made this really painful though, is that there was a person in my class at the university who actually wrote a function like this, but converting from characters to character codes.

Posted in funny, programming | Leave a Comment »

Standalone code using a Rails database

Posted by Daniel Brahneborg on 2007 February 22

I’m using a Rails application to update a database, but I also want to run an external program on that database once in a while. While it might be done by creating a new thread within the Rails application, I wanted a standalone program. However, it should still use the same database configuration, Ruby models, etc.

First I need to load the Mysql driver, YAML parser, and ActiveRecord:

require 'yaml'
require 'rubygems'
require_gem 'mysql'
require_gem 'activerecord'

Then the load path for ‘require’ must be extended to include my models:

$: << File.join(File.dirname(__FILE__), '../../app/models')

To connect to the database, make sure ActiveRecord uses this connection, and that it gets closed afterwards, I use code like this. The application code is sent as a block.

def with_database(config)
  dbh = nil
  begin
    dbh = Mysql.real_connect(config["host"],
      config["username"], config["password"], config["database"])
    ActiveRecord::Base.establish_connection config
    yield
  ensure
    dbh.close unless dbh.nil?
  end
end

First I load the database config. The “development” profile is hard coded, but could easily be changed into using an environment variable or something.

filename = File.join(File.dirname(__FILE__), '../../config/database.yml')
dbconfig = YAML.load(File.open(filename))
dev = dbconfig["development"]

Then I can run the application code:

with_database(dev) {
# yadda yadda
}

When the function completes, either normally or by raising an exception, the database is closed.

Posted in programming, rails | Leave a Comment »