Basic tech stuff

Programming and Linux administration

Never fetch data from views

Posted by Daniel Brahneborg on 2007 August 22

Jeez, when am I going to learn?

I know very well that all data that should be displayed in view must be fetched by the controller.

View must never, ever, under any circumstances, fetch data by themselves.

Still, I manage to break that rule all too often. “It’s just going to fetch THIS little piece of data”. This not only includes SQL statements, but even all kinds of accessors that do more than fetch complete objects from a hashtable or similar.

A while ago I wrote a little find_safely() helper method for my ApplicationController, that handles “Record not found” errors. All find() accesses went through that method, which meant that when the database had to be separated into different systems, which meant that most tables had to have a system_id column, only that single method had to be changed. Yes, the value had to be set in a few places, but it never fetched data for the wrong system.

Unfortunately some views fetched data themselves, which meant that I had to have the system_id check in both ApplicationController and ApplicationHelper. This quickly got out of hand, so it didn’t take long until I moved all those accesses back to the proper controller. When this was done, it was also possible to test that the views were showing the correct data. The normal Rails test harness doesn’t have access to the data that the views fetch, which means it can’t be tested. After the refactoring, it could.

This brings back another of my favourite arguments:

You should write tests, not only to verify that the application does what it is supposed to (you’ll find that out soon enough anyway), but because it forces you to build the application in a better way.

Anyway, the “data from views” problem came back to bite me once again today. The application stores a list of the most recently accessed business objects of each type. This was tested by directly checking the contents of the array where they were stored.

It worked fine, until the objects grew too large, overflowing the “data” column in the sessions table, which in turned caused a “marshal data too short” error too appear in the Mongrel log file. Most pages recommended using a larger column type such as mediumtext, but this is of course a bad idea. Having the poor application read and parse several hundred KB or more on each request, only to serialize it back and update the database afterwards kills performance. A couple of KB is ok, but the 64KB limit of the “text” type is there for a reason.

The proper solution is of course to only store the object id’s, and instanciate the objects only when needed. The application worked again, but now the hard coded test cases started to break. To get them to work, the controllers had to fetch this data, which in turn meant that the views didn’t have to.

Instead of an ugly Module that was imported by both ApplicationController and ApplicationHelper, the “latest objects” code could be merged back into ApplicationController where it belong. The code became simpler, the response time decreased, the database errors disappeared, and a larger part of the code got proper unit tests. Sure, just using “mediumtext” would have been faster in terms of work hours, but the code quality would have suffered.

Andra bloggar om: , , .

2 Responses to “Never fetch data from views”

  1. […] skriva automatiska tester Undrar hur många gånger jag ska behöva inse varför det är så viktigt att skriva automatiska tester till sina applikationer? Det är så himla lätt att tycka “asch, det behövs inte den här […]

  2. […] Never fetch data from views […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: