Ruby

4
May 12

Our Git development workflow

Over the last couple of years we’ve been tweaking our development workflow and it has been really successful for us. It certainly isn’t revolutionary (it is heavily based on the Linux kernel), but it has worked so well and our code quality has improved so considerably that I feel it’s worth writing about. We are a very small team – typically 2 or 3 developers per-project – but I imagine this workflow will only get better at scale. A happy side-effect is that all of our projects are completely setup for remote working. I’m really loving this workflow.

Each project has a maintainer, who is also a developer. This role is simply to be the gatekeeper of origin/master of the Git repo. The maintainer is responsible for checking that a patch has the appropriate tags and is ready for inclusion (see the requirements in point 6 below). These concrete rules (which we do break, occasionally) force us to code review. Contrary to my expectations, I really enjoy the constant review process, from both sides.

We create a new local branch for every feature or bugfix. The workflow of a project is focussed around a PROJECT-devel mailling list. This is where all the patches are posted, reviewed and commented on before being pushed. The reviews are without doubt the biggest win here. We have adopted a strict “no change too small” policy – even a misplaced comma in a comment should be picked up on. This can feel pretty harsh at first, but I’ve found that we soon stop being precious and end up writing much better code anyway. Also, because we’re absolutely sure our commit messages will be read before even touching the repository, they have become a lot more detailed and actually useful.

For the nitty-gritty details, here’s the commit documentation from our projects in full:

doc/commit_workflow:

Commit workflow
===

This project is run in a similar vein to the Linux kernel. You will need to
subscribe to <PROJECT-devel@lists.example.com>.

The project is currently maintained by Mark Somerville <mark@example.com>.

The origin/master branch should _always_ be in a deployable state (and almost
certainly be what is actually deployed).


Some notes
===

* Most code changes should come with appropriate accompanying tests. This
  obviously doesn't apply to things like typo's and there are some other things
  where testing doesn't make sense.


Tags
===

Patches should come appropriately tagged. There are various tags that _should_
be used and others which aren't always required/appropriate.

* Signed-off-by: is the main one. There should be at least one of these on all
  patches by at least the author. Anyone else who handled or was involved in
  the patch or was in its delivery path should also add their Signed-off-by:
  e.g the committer.

  Signing-off a patch is an explicit acknowledgement that you know where it
  came from and that it is, to the best of your knowledge, eligible for
  inclusion in the project.

  The tag would look like:

  Signed-off-by: Mark Somerville <mark@example.com>

* Reviewed-by: says that the reviewer has performed a code review and that the
  changes make sense, look sane and are generally appropriate for inclusion.

  The tag would look like:

  Reviewed-by: Mark Somerville <mark@example.com>

* Tested-by: says that the tester has applied the patch and at least run all of
  the tests using whatever testing makes sense for the given context - it could
  be running `rake` or looking at it in a browser. More thorough tests can be
  performed, of course (and probably should be in some cases). The tag should
  relevant software and versions. For example, it will often include the Ruby
  interpreter and version used.

  We want to support at least MRI (Matz's Ruby Interpreter, the "standard"
  Ruby), version 1.8.7. Some interpreters are below:

  * MRI 1.9.3
  * MRI 1.8.7
  * RBX 1.8.7
  * JRuby 1.9.2

  The tag would look something like:

  Tested-by: Mark Somerville <mark@example.com> [MRI 1.9.3, RBX 1.8.7]
  Tested-by: Mark Somerville <mark@example.com> [RHEL5, MRI 1.9.3, Rails 3.0.11]

* Co-authored-by: is applied to patches that were written by more than one
  author. One example is pair-programming, where the code was created on one
  workstation. In this case the Git Author metadata is set to the owner of the
  workstation and this author would add a Signed-of-by tag, as usual. The other
  developer would add both a Co-authored-by and a Signed-off-by tag.

  The tag would look like:

  Co-authored-by: Mark Somerville <mark@example.com>


Tag ordering
===

Developers can and should add multiple tags to patches. The basic rule is that
a single developer's Signed-off-by tag should come after his others. If other
developers add tags to the patch, they should be added below.

Example: Mark was the author and Euan was the maintainer at the time:

  Tested-by: Mark Somerville <mark@example.com> [MRI 1.9.3]
  Signed-off-by: Mark Somerville <mark@example.com>
  Tested-by: Mike Lauder <mike@example.com> [MRI 1.8.7]
  Reviewed-by: Julia Allison <julia@example.com>
  Signed-off-by: Euan Maxwell <euan@example.com>

Example: Mark and Julia co-authored the patch on Mark's workstation and Euan was
the maintainer at the time:

  Tested-by: Mark Somerville <mark@example.com> [MRI 1.9.3]
  Signed-off-by: Mark Somerville <mark@example.com>
  Co-authored-by: Julia Allison <julia@example.com>
  Signed-off-by: Julia Allison <julia@example.com>
  Tested-by: Mike Lauder <mike@example.com> [MRI 1.8.7]
  Signed-off-by: Euan Maxwell <euan@example.com>

An exception to the Signed-off-by tag being at the end of a developer's tags
but above some others is when the author is also the project maintainer. In
this case, the author's Signed-off-by is the final tag, because he was the last
person to touch the patch (by committing it).

Example: Mark authored the patch and was also the maintainer at the time:

  Tested-by: Mark Somerville <mark@example.com> [MRI 1.9.3]
  Reviewed-by: Julia Allison <julia@example.com>
  Tested-by: Mike Lauder <mike@example.com> [MRI 1.8.7]
  Signed-off-by: Mark Somerville <mark@example.com>


Tag comments
===

On occasion, a comment should be added to a tag. Probably the most common use
of this is when the maintainer is ready to push the patch, but wants to perform
some minor tidy-up first (whitespace fixes, commit message alterations).
Because the change is minor this is easier than re-posting to the list.

The comments should be above the tag that is commented on and be of the form:

  [ mark@example.com: just a wee comment, probably no more than a line ]

An example:

  Tested-by: Julia Allison <julia@example.com> [MRI 1.9.3]
  Signed-off-by: Julia Allison <julia@example.com>
  Tested-by: Mark Somerville <mark@example.com> [MRI 1.9.3]
  [ mark@example.com: whitespace and refactored to idomatic Rails ]
  Signed-off-by: Mark Somerville <mark@example.com>


Merging a change
===

1. When you have made a change that you are happy with, you should commit it to
   your local Git repo with a Signed-off-by tag (see notes above). For commits
   involving code, you probably want to attach a Tested-by tag too.

   This is a good note about commit messages:

   http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html

2. Generate a patch, probably using `git-format-patch` (or `git-send-email`
   directly).

3. Email the patch to the PROJECT-devel mailing list (git-send-email is good
   for this) with the subject: "[PATCH] First line of commit"
   (`git-format-patch` will do this for you).

4. When your feature/fix spans multiple commits, you should probably email a
   patchset. This consists of a summary email with a subject like:

     "[PATCH 0/N] Add a teleporting time machine"

   with the commit patches as separate emails in reply to the summary.
   `git-send-email --compose` is very useful for this.

5. Developers comment on the patch(set). Sometimes this will be as simple as
   adding some tags to the commit. Multiple tags by different developers are
   fine (and welcomed).

   Other times, some changes will be needed before the team agree the patch is
   acceptable. If this is the case, you should incorporate the changes and go
   back to stage 2. The difference this time around is that when posting the
   patch during stage 3, the start of the subject should be changed to
   [PATCH v2] to indicate that this is the new version of the patch to review
   and test.

   You should also update the patch changelog. This versioning belongs below
   the commit message and above the diff stat (just below the ---). An example:

     Wrote pseudo-code algorithm for predicting lottery numbers

     Signed-off-by: Mark Somerville <mark@example.com>
     Co-authored-by: Euan Maxwell <euan@example.com>
     Signed-off-by: Euan Maxwell <euan@example.com>
     ---
      v2 - Added better comments clarifying the algorithm

      doc/lottery_number_prediction | 154 +++++++++++++++++++++++---------------
      1 files changed, 125 insertions(+), 29 deletions(-)

   When sending this v2 patch, be sure to email the patch as a reply to the
   appropriate patch thread, using the Message-ID you want to reply to.

6. To be acceptable for pushing to master, a commit must have at least one
   Signed-off-by tag (the author) and one or more Reviewed-by/Co-authored-by
   tags. There must also be at least one Tested-by tag that is by someone other
   than the original author (this is to protect against things only working due
   to a quirk of a single development or testing machine). Tested-by tags can
   be omitted for commits that don't need any testing or where it doesn't make
   sense, like documentation changes.

7. Once the patch is acceptable, it is the maintainers responsibility to apply,
   amend the commit to add the extra tags and push it to origin.

2
Apr 12

Limiting IRB output

Below is a snippet from my .irbrc file on our production servers, which we access over SSH. It has saved so much frustration by truncating IRB output to 3000 characters. It will, of course, potentially break any code using printf…

module Colours
  Reset = "\e[0m"
  Red = "\e[0;31m"
  Green = "\e[0;32m"
  Yellow = "\e[0;33m"
  Blue = "\e[0;34m"
  Magenta = "\e[0;35m"
  Cyan = "\e[0;36m"
  White = "\e[0;37m"
  BrightRed = "\e[1;31m"
  BrightGreen = "\e[1;32m"
  BrightYellow = "\e[1;33m"
  BrightBlue = "\e[1;34m"
  BrightMagenta = "\e[1;35m"
  BrightCyan = "\e[1;36m"
  BrightWhite = "\e[1;37m"
end

# Only print the first 3000 characters using printf().
#
# It would be nicer to only do this for instances of IRB::Irb, but I can't work
# out how to do that in .irbrc or files required there.
module Kernel
  alias_method :old_printf, :printf

  def printf(*args)
    if args.last.length > 3000
      args.last.slice! 3000...args.last.length
      args.last << "#{Colours::BrightCyan} ...\n  ... etc#{Colours::Reset}"
    end
    old_printf *args
  end
end

I don't understand why something like the snippet below isn't the default:

require "bigdecimal"

class BigDecimal
  def inspect
    "#{Colours::BrightMagenta}#{to_s}#{Colours::Reset} (BD)"
  end 
end

8
Dec 11

Compiling 32bit Ruby on a 64bit machine

We achieved a reasonable improvement in out test suite run time (~10%) by running a 32bit Ruby 1.8.7 rather than 64bit. I’m not sure why. These instructions are for F16.

You need some 32bit libraries:

yum install glibc-devel.i686 openssl-devel.i686 zlib-devel.i686

Unfortunately, Ruby 1.8.7-p352 doesn’t seem to enjoy to being cross-compiled (at least on my setup). The patch for bug #5108 fixes this problem for me.

Next, configure the build. The most basic configure command that you need is probably something like this:

CFLAGS="-m32 -O2" LDFLAGS="-m32" CXXFLAGS="-m32" ./configure --prefix=/usr/local/ruby-1.8.7 --target=i686-unknown-linux-gnu

After building and installing, you will probably want to install RubyGems again and will need to rebuild any C extensions you use. Of course, building them will require any associated 32bit libraries. The mysql gem is slightly trickier than some:

yum install mysql-devel.i686
gem install mysql -- --with-mysql-config=/usr/lib/mysql/mysql_config

31
Dec 10

Urchin 0.1.0 released

A few months ago I wanted to teach myself some new shell tricks. One thing led to another and I ended up writing my own interactive shell in Ruby. Urchin exists mostly just for the sake of it, but I’m looking forward to experimenting and implementing some things that aren’t found in other shells.

Urchin does not aim to be POSIX compliant. Finding POSIX shell scripting pretty nasty was one reason I started this project. However, I generally really like the existing standard shells, so many things work in a way you would expect: pipelines, redirections, job control, globbing, environment variables, aliases and tilde expansion are all there. Documentation, not so much!

Urchin recently passed my criteria for a first release – “run as my login shell for a week without crashing or annoying me”. For sure, there are some problems (tab-completion, quoting and escaping issues spring to mind), but I had to draw a line somewhere and just release something.

Trying it out

This still feels very much like a beta release. The chances are you don’t use a shell exactly like me and this isn’t yet a viable Bash/Zsh replacement.

I’ve only tested this release on Linux with GNU Readline. I’d love to hear how it runs on other platforms. Future versions will be tested on more platforms.

Warning: I’d advise testing it out for a while before you get too trigger happy and chsh!

If I haven’t managed to put you off yet, follow the instructions on the Urchin homepage to try it out.


27
Apr 10

Unix signal programming in Ruby

I recently thought I would be able to use Unix signals to solve a problem in a Ruby program I’m writing. It turned out not to be workable, but was a fun journey into Unix signal handling and how they work (or don’t) with MRI.

I’m not a signals expert – corrections and opinions are very welcome! Also, I started to feel a bit out of my depth when patching the longjmp() calls. I’d love to find out more about this stuff.

SignalPhoto by David Blaikie

Brief signals primer

Signals are used to alert processes or threads about a particular event. Synchronous signals are usually the result of errors in executing some instruction (such as an illegal address reference) and are delivered to the thread that caused the error. Asynchronous signals are external to the execution context and are probably the ones you’re more familiar with – they can be sent between processes using things like kill or delivered when needed by the kernel.

When a signal is generated it is immediately put into the “pending” state. If the process has a thread that has not blocked signals of that type, it is delivered straight away. If that type of signal is blocked by all threads in the process, it remains pending until they are unblocked in one of the threads, at which point it is delivered immediately. Delivered signals can be ignored (often the default response) or processed by a signal handler. In Ruby, we define a SIGUSR1 handler like this:

Signal.trap("USR1") do
  puts "USR1 caught"
end

Why might we want to block signals?

Blocking signals is often used when we have a section of code that must not be interrupted. To enable this, each thread maintains a signal mask. This is the list of signal types that the thread is blocking, which we can examine and change using pthread_sigmask() (sigprocmask() in single-threaded programs). A new thread inherits the signal mask from the parent. However, each thread does not have its own set of signal handlers – these are shared throughout the process. Asynchronous signals that are delivered to the process can be processed by any thread that has not blocked those signals.

Signals in MRI

Unfortunately, MRI isn’t really very friendly to Unix programmers wanting to play with the signal mask, as we’ll see.

MRI defines the Signal module, that only contains two methods: Signal.trap and Signal.list, which provides the mapping of signal names to numbers for your platform. Since none of the other libc signal handling functions are defined, I created a library to provide them (and some other system calls sometime). syscalls is built using the lovely FFI library. This mirrors the libc functions closely, with a couple of Ruby-style shortcuts added.

Ruby 1.8

As Joe Damato found, MRI 1.8 with pthreads enabled is rather rt_sigprocmask() happy. It seemed obvious that all that mucking about with the signal mask would cause strange behaviour when blocking signal, but let’s see how:

require "syscalls/signal"

mask = Syscalls::Sigset_t.new.to_ptr
Syscalls.sigemptyset(mask)
Syscalls.sigaddset(mask, "USR1")

puts "Block and roll!"
Syscalls.sigprocmask(Syscalls::SIG_SETMASK, mask, nil)

puts "Looks fine so far - let's raise an exception..."

begin
  raise
rescue
end

puts "Aw-naw!"

What’s going on?

Using strace with ruby 1.8.6 (2009-08-04 patchlevel 383) [x86_64-linux] gives us:

write(1, "Block and roll!\n", 16Block and roll!
)       = 16
rt_sigprocmask(SIG_BLOCK, NULL, [], 8)  = 0
rt_sigprocmask(SIG_SETMASK, [USR1], NULL, 8) = 0
write(1, "Looks fine so far - let's raise "..., 48Looks fine so far - let's raise an exception...
) = 48
rt_sigprocmask(SIG_BLOCK, NULL, [USR1], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [USR1], NULL, 8) = 0
write(1, "Aw-naw!\n", 8Aw-naw!
)                = 8

Line 3 is from Ruby calling getcontext – it is pretty harmless since it is passing SIG_BLOCK along with an empty set of signals, which adds nothing to the existing signal mask. Line 4 is our own call to sigprocmask – note we’re using SIG_SETMASK, which replaces the existing mask. So far, so good. However, on lines 7-9 Ruby stores the old mask (our SIGUSR1), replaces it with an empty mask and then immediately replaces that with our SIGUSR1 mask again.

But, the mask is only empty for a fraction of a second – I think I’ll be alright!

Think again! It’s tempting to think that this wouldn’t be a problem in most real-world situations, but you may recall that when a signal cannot be delivered because it’s blocked it is put into a pending state. When that signal type is unblocked, the signal is immediately delivered. This means there can actually be plenty of time to queue up a signal to cause a problem here.

The REE stuff below all applies to the other MRI 1.8 flavours I tested too – that’s 1.8.{6,7} on 64-bit Linux.

REE 1.8.7-2010.01

REE has Joe’s --disable-ucontext patch applied, which meant a lot fewer sigprocmask()s to wade through! In fact, it nearly worked – just our old SIG_SETMASK friend set during the exception handling:

write(1, "Block and roll!\n", 16Block and roll!
)       = 16
rt_sigprocmask(SIG_SETMASK, [USR1], NULL, 8) = 0
write(1, "Looks fine so far - let's raise "..., 48Looks fine so far - let's raise an exception...
) = 48
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
write(1, "Aw-naw!\n", 8Aw-naw!
)                = 8

Time to dig around and see why Ruby is doing that.

Calling raise resulted in a call to rb_longjmp(), which appears to be a reimplementation of siglongjmp() (or _longjmp() – I don’t know which). This in turn calls rb_trap_restore_mask(), which sets the signal mask back to the mask that was stored when Ruby starts up or the last call to Signal.trap was made.

Patching Ruby 1.8 – including 1.8.6, 1.8.7 and REE

This might be dangerous or not even sensible. Let me know if you find out!

As far as I can tell, simply removing the call to rb_trap_restore_mask() shouldn’t break anything since the places that the trap_last_mask variable is set are very limited. It may not the best place for the fix (if rb_longjmp() is actually siglongjmp(), this might break the reimplementation), but it does at least appear to work.

Here’s the truly tiny patch.

Ruby 1.9

Ruby 1.9.1-p376 is a slightly more tricky case. As you’ll be aware, MRI 1.9 maps each Ruby thread to a native C thread and uses the GIL to ensure only one runs at any one time. The interpreter uses a thread to trigger an interrupt in order to schedule threads. This thread is created on initialisation of the interpreter and means that even very simple programs have two native threads running.

As we can see in the abbreviated strace below, the signal mask is empty when the timer thread is created and this will be inherited. This means that if we block a signal type in our main Ruby thread, they will still be able to be delivered and handled by the timer thread.

Below that we can see rb_trap_restore_mask() emptying the mask when it is called from rb_longjmp(). The sigaltstack() and following sigaction()call on lines 4-5 tell Ruby to handle segfaults on a different stack.

rt_sigaction(SIGHUP, {0x48b9f0, [], SA_RESTORER|SA_SIGINFO, 0x3deca0f0f0}, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGUSR1, {0x48b9f0, [], SA_RESTORER|SA_SIGINFO, 0x3deca0f0f0}, {SIG_DFL, [], 0}, 8) = 0
...
sigaltstack({ss_sp=0x1b0baf0, ss_flags=0, ss_size=16384}, {ss_sp=0, ss_flags=SS_DISABLE, ss_size=0}) = 0
rt_sigaction(SIGSEGV, {0x48bce0, [], SA_RESTORER|SA_STACK|SA_SIGINFO, 0x3deca0f0f0}, {SIG_DFL, [], 0}, 8)
...
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
...
clone(Process 23020 attached
child_stack=0x7f3b2c188ff0, flags=CLONE_VM|CLONE_FS| CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM| CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f3b2c1899e0, tls=0x7f3b2c189710, child_tidptr=0x7f3b2c1899e0) = 23020
...
[pid 23019] write(1, "Block and roll!", 15Block and roll!) = 15
[pid 23019] write(1, "\n", 1
)           = 1
[pid 23019] rt_sigprocmask(SIG_SETMASK, [USR1], NULL, 8) = 0
[pid 23019] write(1, "Looks fine so far - let's raise "..., 47Looks fine so far - let's raise an exception...) = 47
[pid 23019] write(1, "\n", 1
)           = 1
[pid 23019] rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid 23019] write(1, "Aw-naw!", 7Aw-naw!)      = 7
[pid 23019] write(1, "\n", 1
)           = 1

Patching Ruby 1.9.1

First off, we need to mask all signals as soon as the timer thread is created. This makes sure that all signals can only be delivered to the main Ruby thread (until we create some more). I suppose this will actually slow down signal delivery by some small amount.

Ruby itself blocks some signals for a time to allow sections of code to run without interrupts, using rb_disable_interrupt() and rb_enable_interrupt(). These functions mask and unmask all signals. We need to make Ruby save the existing signal mask while it blocks all signals, then restore the old mask, rather than unblocking everything.

Here’s the patch. Again, I don’t think this has any negative side-effects, but it also might not be a good idea. Let me know if you find out!

Further reading


27
Mar 10

Ruby process management with Jobby

Jobby is a generic forking process manager written in Ruby, built with robustness as a primary goal from the start. It was initially developed for offloading long running background tasks from the webserver in Rails applications, but it turned out to be useful in its own right, so we extracted it to work more generally. It uses Unix sockets and a call to fork(), so won’t work on Windows or JRuby, but has been tested on several flavours of Linux and OS X. It is also copy-on-write friendly, should the Ruby interpretter that’s used support it.

If you prefer to read code than blog posts, server.rb does most of the heavy lifting.

After a long struggle of using the old BackgrounDRb, last year we finally threw in the towel. We checked out the alternatives, but nothing really fitted the bill. Since background job processing is quite central to some of our applications, we built our own. From the start we had some requirements:

  • Run jobs in parallel up to a maximum number or processes, then start queuing
  • Robust – it just had to be solid
  • Easy to manage – we hated those BackDRb scripts
  • Reliable logging
  • Able to get messages from the background processes

Jobby itself does not provide a mechanism for getting messages from the child processes. I’ll detail a jobby_rails plugin that handles that some other time.

Installing Jobby

gem install jobby will do the trick if for some reason you don’t hate gems. You can also check out the source from github.

Gentoo users can install Jobby from the Gentoo Ruby overlay. Note: I stopped using Gentoo sometime around March 2010. It might still be there.

How does it work?

Jobby is a command line program. It consists of a daemon that listens on a Unix socket and will fork when it receives a connection. The forked child either runs some Ruby code or executes some shell code, depending on how the daemon was started. When you call Jobby, you pass a single parameter to STDIN – this string is passed to the forked child. It could be an ID of some sort, a whole email, a marshalled object or whatever.

There is only one command used, jobby. When this is run it first checks for the existence of the daemon process listening on the socket. If it’s not there, it starts one (and then executes the child command). This is great because it makes daemon startup totally automatic and if the daemon should go down (this hasn’t happened to us yet in the months that we’ve been using it), the next request will start up a new one.

Stopping Jobby

There are two ways to stop the daemon, both using Unix signals. Issuing a USR1 initiates a “very friendly shutdown”, which will stop the daemon accepting any new connections but will allow it to finish the jobs in the queue before terminating. Issuing a TERM signal will stop the daemon forking, then issue a TERM signal to the child processes and terminate the daemon.

Other nifty stuff

When using Jobby to run Ruby code, you might want to have the children load a few libraries before they do any work. Repeating this every time is a pain, so you can instead pass a --prerun parameter to prerun some Ruby code in the daemon before any forking takes place. This will load the libraries (at least in this example) so that they are immediately available in the children when the fork occurs. If you’re using a copy-on-write friendly interpretter, like Ruby Enterprise Edition, you’ll get the memory saving benefits of that too.

You can run multiple Jobby daemons – just call each one by passing a different socket parameter. This is how you can run different code for different job types that you have.

To make your debugging life easier, the forked children will show up in process listings as jobby: /path/to/socket. The actual daemon process will show up as jobbyd: /path/to/socket.

Using Jobby with Rails

Jobby was designed for use in Rails applications, but was extracted since it is generally useful. This post only describes Jobby itself, which you can happilly use from a Rails app using a system call. Sometime I’ll get round to releasing and detailing a Rails plugin for Jobby that adds a communication layer to Jobby and makes Rails intergration nicer. We’re using it just now, but it’s not really releasable as it is. I decided that since the main Jobby code has been so stable for us, it made sense to release it now.


2
Apr 08

Nibbles 0.1 released

I’m pleased to say that I’ve just released version 0.1 of Nibbles, an alternative RSS and Atom feed reader for Maemo. A big thanks to the people who were kind enough to help out with the pre-release testing, you’ve been (and hopefully continue to be!) a great help.

Nibbles screenshot

Nibbles is quite stable, can parse a wide variety of feeds and is pretty bandwidth efficient. Nibbles is finger-friendly and does aim to be a good mobile feed reader, but things like downloading of full offline articles didn’t make it into this initial release (I have some working code for that, but it’s not working enough!). New users might want to know that the zoom in/out hardware buttons hide and show the feed and article lists.

Of course, since this is a first release, things aren’t perfect. The biggest problem for me that I think people should be aware of is that Nibbles uses too much battery power. This is detailed in bug #2342 and is really a problem with ruby-gtk2 rather than Nibbles. As I said, with some luck, the next ruby-gtk2 release will fix this.

You can install Nibbles from the project page.


27
Mar 08

Does anyone have the ruby-maemo source?

Tom Swindell (AKA alterego) created the Maemo bindings for Ruby and they’re great. I’m writing a couple of programs with them, including Nibbles, my feed reader. Unfortunately, Tom seems to have vanished and I’ve not been able to find the source code for the bindings anywhere. Does anyone have it? I’d really like a copy.

In other news, I hear that a new release of ruby-gtk2 is near. This is excellent as it will hopefully fix the battery life problem with the current bindings where Ruby/GTK programs (and, therefore, Ruby/Maemo programs) wake up 100 times a second regardless of what they’re doing. Happy days.


15
Feb 08

Nibbles, a feed reader for Maemo

I don’t really get on with the supplied RSS reader on my N800. It doesn’t work like I expect and it lacks mobile features like offline reading that I’d like. So, I’m writing a feed reader for Maemo called Nibbles. It’s still very early stages, but there’s enough for a development release now, I think. Below is a screenshot of the current preview version, 0.0.4:

Nibbles screenshot

It seems to cope nicely with a wide variety of RSS and Atom feeds, despite my proof-of-concept code somehow making it into the app ;).

Currently it’s quite bare of features and can be considered the simplest app that could work (for example, the feeds update at a fixed 30 mins). It will likely stay that way until I get a few bugs worked out and 0.1, the first ‘stable’ version, released. After that, I’ll work on some more interesting new features. I’m still very open to suggestions though and all feedback is welcome, so please don’t be shy!

If you’d like to play along and help the development of a better feed reader for your internet tablet, use the .install file. Please report issues to the Maemo Garage tracker.

You can make Nibbles the default feed reader by editing /usr/share/applications/defaults.list. You need to set hildon-news_reader=nibbles.install. This will allow you to tap on the feed icon in a browser window and have the feed added to Nibbles.

I’ve only specified that the app will work in OS2008, but I’m pretty sure it’ll work in OS2007 too. Since I only have an OS2008 machine, I can’t test it easily. If anyone is still using OS2007 and would like to help, please do get in touch.


21
Jan 08

Creating RubyGems .debs for Maemo

I’m writing an application (more on that sometime soon) using the very nice Ruby bindings for Maemo. I’ve got some dependencies that are packaged as RubyGems that I wanted to be managed through the Application Manager, so I created some .debs for them. Here is their temporary home The rake, hpricot, feed-normalizer, simple-rss, atom and htmlentities gems are available in my repository. I didn’t create the rubygems and librubygems packages themselves – I was sent them, but couldn’t find them hosted elsewhere.

It took me a little while to suss out how to create the .debs so I’ve included the steps here in case anyone finds themselves in the same boat. This works for native Ruby gems and compiled C extensions.

  1. Install the gem inside Scratchbox. Irritatingly, “gem install example” doesn’t always want to do very much, so you may need to dowload the gem file and “gem install example-0.1.gem”.
  2. Make a new directory with the rubygems-<package>-<version> syntax (the rubygems- prefix seemed to make sense to me): mkdir rubygems-example-0.1
  3. cd rubygems-example-0.1/ ; mkdir gems specifications bin
  4. cp -r /targets/CHINOOK_ARMEL/var/lib/gems/1.8/gems/example-0.1/ gems
  5. cp /targets/CHINOOK_ARMEL/var/lib/gems/1.8/specifications/example-0.1.gemspec specifications/
  6. cp /targets/CHINOOK_ARMEL/var/lib/gems/1.8/bin/example bin/ (if there are any bins)
  7. Create a Makefile with this content (you will need to replace the spaces at the start of the lines with tabs):
    all:
    clean:
    install:
            mkdir -p $(DESTDIR)/var/lib/gems/1.8/gems/
            mkdir -p $(DESTDIR)/var/lib/gems/1.8/bin/     <------ REMOVE THIS LINE IF THERE ARE NO BINS
            mkdir -p $(DESTDIR)/var/lib/gems/1.8/specifications/
            cp -r gems/* $(DESTDIR)/var/lib/gems/1.8/gems/
            cp bin/* $(DESTDIR)/var/lib/gems/1.8/bin/     <------ REMOVE THIS LINE IF THERE ARE NO BINS
            cp specifications/* $(DESTDIR)/var/lib/gems/1.8/specifications/
  8. dh_make -e you@example.com then choose the type of package.
  9. Edit debian/control to suit. Make sure the section is something like user/Utilities, not just Utilities if you want the package to show in the Application Manager.
  10. dpkg-buildpackage -rfakeroot -b
  11. The parent directory should now contain rubygems-example_0.1-1_armel.deb

As a final note, /var/lib/gems/1.8/bin/ may not be in $PATH, so you'll need to deal with that if you have files in bin/.


28
Dec 07

A Ruby binding to cracklib

Just released rubylibcrack as a gem (0.1.1). I developed this teeny-weeny binding to the *nix password strength checking library cracklib at my work. Big props to my work, they totally get open source and the way things should be done.

The gem has only been tested on Redhat, Fedora, Gentoo and Ubuntu (all 64bit, I think), but I’d love to hear how it fares on other platforms. Here’s a Gentoo ebuild for the package.


9
Apr 06

Random porn stories in Ruby

I’d never done a Ruby Quiz challenge before, but the latest one sounded like especially good fun. At a party after clubbing the other night, Jason told us that his ex-girlfriend used to write dirty stories for porn magazines for a couple of hundred quid a pop. This was the final inspiration I needed.

I wrote a Markov Chains algorithm and although it’s a straightforward problem, it was quick and really good fun. I won’t bother posting my code here, it’s quick and dirty (and it’s all lowercase, but I’ve been partying this weekend and my short attention span for such things has run out) – search the Ruby-talk mailing list in a few days (once people are allowed to post their answers to the quiz) and there will be loads of excellent ideas there.

Of course, it will generate any old crap, but I’ve only fed it porny stories from Google, because I can’t think how I could get funnier output. I tried mixing in some emails from my friends, but that didn’t work out as well as I’d hoped. Still, naughty words are cool.

OK, now cue a load of hits from the search engines :). Here are a couple of short ones, around 250 words (a nasty hack prevents me getting an exact number of words shrug):

they were lovers from afar, meeting clandestinely almost everyday. one lived in new england, while the other somewhere in the game and each sister found implements suitable for such purposes; a pencil or a pen. at times, a neighborhood friend would be asked to join in the back of her husband. winnie described must be extremely careful to maintain their secrecy from the two young women and a faint feeling came over her. she began to develop between her legs.winnie, with difficulty, began to insinuate into the jelly and scooped a dab. painstakingly, she drew her middle finger at her computer and with trembling hands, she managed to pull down her panties in order for winnie to take a deep breath and hold it. at the keys for her immediate reply to christina. this time, she would have more time, she pulled it out completely and timidly brought it to access that story. she sat back in her tummy. one particular story caught her attention. scanning down the length of the threesome that played doctor in their secret place. after fingering her rear passage for sometime, her arm began to re-read those mesmerizing words.as the three little girls were playing doctor and taking each other’s temperatures, winnie put herself in her tummy.

anne abandoned herself completely to the soft breeze, and naked exposure titallating. she would sometimes masturbate on the closest patio chair, revealing as the shock dissipated, that she was more fair, and her face lit up in a circular fashion her clitoris, first slowly, and then more rapidly. the small lips surrounding her vaginal opening were pulled rhytmically as the shock dissipated, that she was also turned on. jessica’s moistened body was covered in sweaty moisture over its entire length. anne’s eyes were still closed. her rapid breathing becoming mild moans and sighs. anne abandoned herself completely to the back of her breast, kneading avidly while twisting and abusing the nipple. pleasure rolled in waves through her body, emanating from her breasts and thighs. she smiled at jessica’s flirtations, tracing with her left hand grabbing the zipper on anne’s shorts, pulling the zipper down. anne stood frozen. she could now see a wedge of dark pubic hair, then, her buttocks returning to the back of her secluded mansion. she enjoyed sunbathing nude alone in her face. her complexion was more fair, and her body soft and voluptious, compare to the soft curves of the body before her and jessica.

Edit: I’m sure many people have done this already – it would be cool to use Markov Chains to write a program to generate random techno music. I don’t really know anything about music but I think the same basic principles should apply (although I doubt you could knock it together in a few hours on a Sunday afternoon). I’ll file that one under the “when I win the lottery” category.

The lack of spaces after the full-stops are thanks to the original authors!