Basic tech stuff

Programming and Linux administration

Archive for August, 2007

Never fetch data from views

Posted by Daniel Brahneborg on 2007 August 22

Jeez, when am I going to learn?

I know very well that all data that should be displayed in view must be fetched by the controller.

View must never, ever, under any circumstances, fetch data by themselves.

Still, I manage to break that rule all too often. “It’s just going to fetch THIS little piece of data”. This not only includes SQL statements, but even all kinds of accessors that do more than fetch complete objects from a hashtable or similar.

A while ago I wrote a little find_safely() helper method for my ApplicationController, that handles “Record not found” errors. All find() accesses went through that method, which meant that when the database had to be separated into different systems, which meant that most tables had to have a system_id column, only that single method had to be changed. Yes, the value had to be set in a few places, but it never fetched data for the wrong system.

Unfortunately some views fetched data themselves, which meant that I had to have the system_id check in both ApplicationController and ApplicationHelper. This quickly got out of hand, so it didn’t take long until I moved all those accesses back to the proper controller. When this was done, it was also possible to test that the views were showing the correct data. The normal Rails test harness doesn’t have access to the data that the views fetch, which means it can’t be tested. After the refactoring, it could.

This brings back another of my favourite arguments:

You should write tests, not only to verify that the application does what it is supposed to (you’ll find that out soon enough anyway), but because it forces you to build the application in a better way.

Anyway, the “data from views” problem came back to bite me once again today. The application stores a list of the most recently accessed business objects of each type. This was tested by directly checking the contents of the array where they were stored.

It worked fine, until the objects grew too large, overflowing the “data” column in the sessions table, which in turned caused a “marshal data too short” error too appear in the Mongrel log file. Most pages recommended using a larger column type such as mediumtext, but this is of course a bad idea. Having the poor application read and parse several hundred KB or more on each request, only to serialize it back and update the database afterwards kills performance. A couple of KB is ok, but the 64KB limit of the “text” type is there for a reason.

The proper solution is of course to only store the object id’s, and instanciate the objects only when needed. The application worked again, but now the hard coded test cases started to break. To get them to work, the controllers had to fetch this data, which in turn meant that the views didn’t have to.

Instead of an ugly Module that was imported by both ApplicationController and ApplicationHelper, the “latest objects” code could be merged back into ApplicationController where it belong. The code became simpler, the response time decreased, the database errors disappeared, and a larger part of the code got proper unit tests. Sure, just using “mediumtext” would have been faster in terms of work hours, but the code quality would have suffered.

Andra bloggar om: , , .

Posted in rails | 2 Comments »

Basic’s programming laws

Posted by Daniel Brahneborg on 2007 August 1

I’ve come to realize that the following laws apply when debugging a program.

  1. A program that runs on several platforms will fail on the one with the worst tools for debugging.
    1. Having gdb installed on one machine while being absent on another, makes the bug appear on the second machine.
    2. Having Purify or Valgrind installed on a subset of the platforms, guarantees that the bugs will occur only on the ones without these tools.
  2. With identical toolsets, the bugs will occur on the machine that is hardest to access.
    1. With ssh access to two machines, the bug will occur on the one to which you can’t use keybased login.
    2. If one machine requires Cisco VPN login, which never seems to work correctly, that’s the machine which will fail.
  3. If the program is multithreaded, the bug will only appear when running outside of the debugger, since these tools always affect the relative timing of the threads.

On the other hand, the reason I’m a programmer is not because it’s easy, but because it’s hard.

Andra bloggar om: .

Posted in programming | Leave a Comment »