Lojic Technologies

Posts Tagged ‘rails

Beware of YAML serialization on Linux without libyaml

leave a comment »

The Problem

I noticed some unusual behavior with respect to YAML String serialization between my Linux production system and my Mac OSX development system.

After dumping the production database via pg_dump -O –no-acl mydb | gzip > ~/mydb.sql.gz and then restoring it on my development system via rake db:drop; rake db:create; psql mydb < mydb.sql, I noticed that a particular serialized field in my Rails app that should always be an Array of String objects occasionally contained Integers.

After a little research and experimentation, I discovered that the production Linux system would occasionally omit quotations around Strings containing only numeric digits. I haven’t analyzed the pattern fully, but here are some examples where the YAML serialization did or did not use quotes:

  • “90103”
  • 000080
  • “000071”
  • “000124”
  • “000003”
  • 008397
  • 000408
  • 000009
  • 000188
  • “000021”

Further investigation revealed that the Linux production system was using Syck (a “dated C implementation of YAML 1.0”) and my Mac OSX development system was using psych (a “libyaml wrapper (in Ruby core for 1.9.2)”). libyaml is a “fast C implementation of YAML 1.1. So, either the quotation rules have changed between YAML 1.0 and YAML 1.1, or there is a bug in one of the implementations (likely Syck).

The Solution

The solution for proper “future” behavior is pretty simple. Install libyaml on the Linux system as follows:

wget http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz
tar xzf yaml-0.1.4.tar.gz
cd yaml-0.1.4
./configure
make
make install

I think that’s enough, but I went ahead and rebuilt my Ruby 1.9.2 just in case it needed to know about the existence of libyaml at build time.

The solution for converting my database with YAML 1.0 serialization to YAML 1.1 serialization is a bit trickier. Since the “dump” and “load” operations are matched for a particular version of YAML, it seems difficult to load the data using YAML 1.0 (thereby retaining the String type when reading an unquoted 000088) and then dump the data using YAML 1.1 (to get proper quoting of ‘000088’). Further complicating this is the fact that Rails handles the serialization operations automatically.

It does appear possible to dynamically switch between syck and psyck by using the following:

YAML::ENGINE.yamler = 'syck'
YAML::ENGINE.yamler = 'psych'

So, one option is to repeatedly switch to syck, read in data, switch to psych, and then write the data. <sigh>

Update:

It appears that due to the semantics of the Rails serialize function, it’s not enough to just read the model object using syck and then immediately write with psych because that doesn’t appear to be enough to cause the field to be deserialized. I had to refer to the field for each object. This is a pain because it prevents me from doing a generic loop where I can handle all model objects easily w/o reference to their specific fields.

I’ll withhold judgment for a while, but my first inclination is to consider abandoning YAML serialization for something a little more robust and portable.

Update 2:

It appears my welcome from psych is a serious memory leak. I’ve been running long running Ruby/Rails processes for years, and this is the first time I’ve experienced a failure due to an out of memory condition. There are a number of Google hits regarding the issue. After I fix the leak, I’ll begin researching alternatives to YAML serialization in Rails.

Update 3:

The number of bug reports on psych and rubygems I’ve had to wade through recently is amazing. My current solution is to remove the psych system gem and install Ruby 1.9.3p0 which required upgrading Passenger to the latest version from source to get Ruby 1.9.3 compatibility. I still had to track down a few odd errors such as “undefined method `yaml’ for #<Psych::Nodes::Stream:…>” and “invalid date format in specification: “2011-10-02 00:00:00.000000000Z”” – all because I chose to use the default Rails serialization assuming there would be no issues. Lesson learned.

Written by Brian Adkins

November 22, 2011 at 2:44 pm

Posted in programming

Tagged with , ,

Setup shoulda and rcov for Rails 2.2.2

with 4 comments

I had a few mis-configuration issues when setting up shoulda and rcov for a new Rails 2.2.2 project, so I thought I’d jot down a few notes (mini tutorial, quickstart) to help save others from burning time on what should be a simple task.

shoulda is a library build on Test::Unit that provides helpers, macros and assertions to make testing easier.

rcov is a code coverage tool for Ruby.

1. Install rcov

sudo gem install rcov

2. Install shoulda

sudo gem install thoughtbot-shoulda --source=http://gems.github.com

3. Create your Rails project

rails myapp

4. Modify myapp/Rakefile

require(File.join(File.dirname(__FILE__), 'config', 'boot'))
require 'rake'
require 'rake/testtask'
require 'rake/rdoctask'
require 'tasks/rails'
require 'shoulda/tasks'
namespace :test do
  desc 'Measures test coverage'
  task :coverage do
    rm_f "coverage"
    rm_f "coverage.data"
    rcov = "rcov -Itest --rails --aggregate coverage.data -T -x " rubygems/*,/Library/Ruby/Site/*,gems/*,rcov*""
    system("#{rcov} --no-html test/unit/*_test.rb test/unit/helpers/*_test.rb")
    system("#{rcov} --no-html test/functional/*_test.rb")
    system("#{rcov} --html test/integration/*_test.rb")
    system("open coverage/index.html") if PLATFORM['darwin']
  end
end

5. Modify myapp/test/test_helper.rb

...
# Add the following line
require 'shoulda/rails'   # require 'shoulda' also worked
...

Conclusion
After you’ve written some shoulda tests, you should be able to use the following rake commands:

rake test
rake test:units
rake shoulda:list    # display specs from shoulda tests
rake test:coverage   # run rcov and display code coverage

Written by Brian Adkins

February 26, 2009 at 9:10 pm

Posted in programming

Tagged with , , , , ,

Gospel Software, LLC

leave a comment »

Gospel Software LogoAlmost a year ago, I formed Gospel Software, LLC with two friends from my church. Our mission for the company is to develop web applications to help churches administratively.

It’s been a joy to work with Jordan and Scott over the last year and we’re now to the point of beginning to heavily promote our three web applications to churches. These are two men I can learn a lot from – both technically and spiritually. I would be hard pressed to find better business partners.

Gospel Software currently has three web applications.

Directory:

I just released a new version of the Gospel Software Directory a few minutes ago – there are some new screen shots to show some of the features. I had wanted a nice online photo directory for our church for quite a while. I finally wrote a simple bare bones version a few years ago and ended up using it all the time, so I thought there might be a market for the product.

Over the last year, I wrote a completely new version, and now each church member can edit their own information, upload new photos, etc., so the information is more current and the church administrative staff has less work to do. I still think one of the best features is simply being able to match the faces of people I’ve met with their names. It’s now available for churches to try out and purchase.

This new version is just the beginning. I have a long list of enhancements I’ll begin rolling out over the next few months.

GuestView:

Scott’s GuestView program is something I use regularly as I follow up with visitors to our church. It’s so handy and easy to use. I get an email when I need to call a visitor, then I can enter notes about our conversation, and if they’d like information from another leader in the church, I can notify the appropriate people.

SongBook:

I had thought about developing a program to manage worship songs back in the mid-eighties, but I was never motivated enough to do anything about it. When Jordan showed me his SongBook application, I was blown away – it did everything I had thought of and much more. And of course it was web based since the internet had been invented since I began thinking of a similar program 🙂

I’m excited about seeing what will be happening with Gospel Software, LLC this coming year.

We do have an affiliate program that rewards both the affiliate and any church they refer. Contact me for details if you’re interested.

The technical experiences we’ve had over the last year should provide for some interesting and informative blog posts in the future. When we came together to form the company, we had three products written in three different languages / frameworks. Integrating the three products together with a common infrastructure has been very educational 🙂

We now have a robust infrastructure that will support any future applications very well.

  • Server configuration, backup & light disaster recovery
  • Ecommerce – credit card processing, subscription management, invoicing, etc.
  • Auditing and event logging
  • Easy deployment of new releases
  • User management, authentication, authorization & accounting
  • And more…

As I mentioned, there are three languages / frameworks involved, but there is quite a bit of Ruby and Rails, and there will likely be more in the future. Each of the three languages / frameworks have their pros & cons, but I do feel that Ruby and Rails does very well in the evaluation.

The following are some things that I have been particularly pleased with:

  • My Macbook Pro with OSX and Emacs as a development environment
  • Ruby & Rails
  • nginx web server
  • mongrel application server
  • Postgres relational database
  • Trac issue tracker & wiki
  • Subversion source code control (possibly moving to git in the future, but for our purposes, svn has worked out very well)
  • Slicehost.com – being able to restart a VM on another server if hardware fails is awesome
  • Ubuntu Linux
  • istockphoto.com and fotolia.com for inexpensive stock photos
  • jQuery – it might not satisfy comp.lang.javascript, but it’s been great
  • Did I mention Emacs? 🙂

Written by Brian Adkins

February 11, 2009 at 2:50 am

Posted in business

Tagged with , , , , , , , ,

Blog Bifurcation

with 8 comments

One of the reasons I haven’t been blogging much lately is because I’ve decided to bifurcate my blog into a professional/technical blog (which will continue here on lojic.com/blog) and a personal blog, and until I’ve decided on the technology to use for my personal blog I’ve been reluctant to blog much.

The motivation for the split is the feeling that a lot of my non-technical family & friends grow weary of weeding through a lot of techno-geek material to find anything interesting, and folks who read my blog for technical info probably don’t want to weed through the silly videos, etc.

WordPress has worked fine for my blog thus far, but I want to take the opportunity to develop my personal blog in a new technology more for the learning experience than necessity. I haven’t had time to select the appropriate technology, so I have a bit of analysis paralysis.

The candidates are:

  • Ruby on Rails: I currently develop primarily in Ruby on Rails, so in that respect it would be the logical choice and easiest way to get started; however, it wouldn’t have the benefit of learning a new technology.
  • Arc: I had high hopes for Arc when Paul Graham first released it. I still think it has potential, but that potential is limited by Paul’s interest level and available time. It’s been over 3 months since the last release and that was only a small incremental improvement. The forum seems dead, and the fact that Arc went through a 5 year blackout period makes me wonder whether it will be a dead-end language and a waste of valuable time.
  • Common Lisp: I am leaning toward a Lisp, so if Arc doesn’t pan out, Common Lisp would be a good fallback language. It’s much more mature with robust implementations. It doesn’t provide a nice batteries included experience though, and I’ve been reluctant to collect the necessary libraries from various sources to allow anything remotely similar to Ruby on Rails with respect to ease of development. I think it may have a greater long term potential though, so it may be worth the effort.
  • Scheme: The PLT web server may give me a head start on a Lisp based web site, and Arc is based on MZScheme, so it’s on the short list.
  • Haskell: I know very little Haskell (even less than Lisp which is not much), but I’m intrigued by many aspects of the language. GHC seems to be a great compiler that produces well performing programs. My initial impression is that it will take more effort to learn than a Lisp, but in terms of brain stretching, it has a lot to offer. There is a Haskell based web server available, but like a lot of fringe languages, it appears to be pretty rough around the edges.

I have a vacation coming up, so I think I’ll use some of the down time to do some research and make a decision. Look for the blog bifurcation to happen in the latter half of June. If you have any opinions on the matter, please add a comment 🙂

Written by Brian Adkins

May 31, 2008 at 3:49 pm

Posted in communication

Tagged with , , ,

Ruby Hoedown 2007

leave a comment »

I just signed up for the Ruby Hoedown conference. It’s being held here in Raleigh at Red Hat headquarters on August 10 & 11, and the cost is only $100. Speakers include Bruce Tate, Marcel Molina Jr., Ken Auer, Ezra Zygmuntowicz (cool name 🙂 ), Andrea O. K. Wright, Jay Phillips & Jared Richardson.

I also signed up for the charity workshop on testing techniques put on by Marcel Molina, Jr., Bruce Tate, and Chad Fowler – the donation proceeds for that workshop go to the Food Bank of Central & Eastern NC. It will probably be a smaller group with more interaction.

Should be an interesting and fun time – gain some knowledge, make some contacts. Let me know if you’re also planning on attending.

Some links:

Written by Brian Adkins

August 1, 2007 at 2:17 pm

Posted in communication

Tagged with , ,

Using helpers inside controllers in Ruby on Rails

with 5 comments

Rails provides some nice helper functions (numbers, dates, etc.) that are available to views, but they’re not automatically available to controllers. I found a number of ways to accomplish this on the web, but I wasn’t satisfied with any of them.

The motivation for me to do this was to use the number_to_currency() helper function in a controller, so I’ll use that for the example. This helper is in the ActionView::Helpers::NumberHelper module, which on my Ubuntu 7.04 system is located here:

/usr/lib/ruby/gems/1.8/gems/actionpack-1.13.3/lib/
action_view/helpers/number_helper.rb

It may be located elsewhere on your system. If you’re running *nix (or OSX), then typing the following command should locate it:

locate number_helper.rb

If you edit number_helper.rb, you’ll notice that NumberHelper is a module and number_to_currency() is an instance method, so we’ll need an instance that has included NumberHelper to be able to call the number_to_currency() method.

One way to do this would be to create a new class that includes the module, and then create an instance of that class:

class MyNumberHelper
include ActionView::Helpers::NumberHelper
end
my_helper = MyNumberHelper.new
formatted_str = my_helper.number_to_currency(n)

If we had an existing object, called obj, another way to accomplish this would be via:

obj.extend(ActionView::Helpers::NumberHelper)

Since I didn’t have an appropriate object to extend the NumberHelper module, I simply created a new one:

Object.new.extend(ActionView::Helpers::NumberHelper)

If I only need to call the helper once, I could accomplish that via:

Object.new.extend(
ActionView::Helpers::NumberHelper).number_to_currency(n)

However, I preferred to create a Proc object and store it for future use:

format_currency = lambda { |n|
Object.new.extend(
ActionView::Helpers::NumberHelper).number_to_currency(n) }

format_currency.call(n)

UPDATE: it’s not necessary to use the lambda (as cool as they are), simply storing the result of Object.new.extend(ActionView::Helpers::NumberHelper) is sufficient. For example:

x = Object.new.extend(ActionView::Helpers::NumberHelper)
x.number_to_currency(7.4)
# results in "$7.40"

END UPDATE

I deliberately wanted to avoid simply including ActionView::Helpers::NumberHelper in the controller to avoid polluting the namespace and possibly expose additional public methods in the controller.

Written by Brian Adkins

July 27, 2007 at 7:20 pm

Posted in programming

Tagged with , ,