PStore Meets YAML

Posted over 2 years ago in The Standard Library.

I love the PStore standard library. It's a very graceful interface to get some fairly robust serialized mini-database handling in just a few lines. With it you get:

  1. Transactions with commit and rollbacks (automatic on exception).
  2. File locking, shared and exclusive.
  3. Multiprocessing safety.

PStore does even more, including some file mode checking and MD5 hashing to avoid unneeded writes, but the above are the major selling points for me.

Now, if I had to level any one complaint at PStore, it would be that because it uses Marshal under the hood it doesn't create files you can easily browse or tweak by hand. (Marshal is a feature, don't get me wrong. It's fast, which is very helpful.) Sometimes though I want PStore protection with the YAML file format.

I'm embarrassed to admit that I use to use a hack for this:

require "pstore"
require "yaml"
class PStore; Marshal = YAML; end

That just redefines the Marshal constant in a scope that should only alter PStore. The library only uses dump() and load() and those methods work the same with Marshal and YAML.

Ready for the punch-line?

I learned today that my fragile hack has been in vain, no matter how clever it may be. YAML ships with a file that will load and modify PStore for you. Usage is as simple as:

require "yaml/store"

From there just replace your PStore.new() calls with YAML::Store.new() and you're in business. YAML::Store is a subclass of PStore, so you won't need to change one bit of the API to get PStore robustness with YAML output.

rahul benegal added 6 months later:

Dear James I have been using ruby for some time and had something to say about pstore and yaml. But first a question:

  1. I am using yaml files for configuration of applications (key-value pairs).

However, just as in our Unix config files we may first mention a variable such as GEM_HOME and then use that later (e.g. $GEM_HOME/bin), i was wanting a user to be able to define a minimum number of settings in the config file. The rest maybe derived from the above.

However, there doesn't seem to be any Yaml syntax for that. Is there some standard solution for the problem, or is my approach of using hashes wrong ?

  1. The second point i might mention is that i tried using YAML to store data structures that contain HTML data. However, the loads would fail cos the HTML would have ":" and many other such characters that YAML could not take care of. (i had tried EscapeHTML but that also did not solve my issue).

Then i discovered Pstore and used it to store the structures, and all is fine. Of course, that means that i cannot view or hand-edit the pstore file. (XML if i recall, has something like CDATA or some syntax for specifying that the following is not to be interpreted until some set of closing characters come.

Thanks a lot. It seems you have been v busy for a longtime. Your blog is a gr8 inspiration for us. Pleaes keep blogging.

rahul b.

James Edward Gray II added 6 months later:

I think Hashes are a fine choice. One way to handle this would be to merge the user settings with an existing defaults Hash:

>> DEFAULT = {:one => "default one", :two => "default two"}
=> {:one=>"default one", :two=>"default two"}
>> user = {:two => "override two"}
=> {:two=>"override two"}
>> settings = DEFAULT.merge(user)
=> {:one=>"default one", :two=>"override two"}

As for YAML generating content it cannot parse, the best thing to do would be to reduce it to a minimal case where it fails and report it as a bug on the Ruby Core mailing list.

Hope that helps.

Add Your Thoughts

You can use Markdown in the body of your comment to format text and make links.

Note that I reserve the right to edit any content you post here. I typically exercise this right to fix formatting issues. All posts must be approved so spam will never been seen on these pages.

Author:
URL or Email (optional):
Body: