fidzu : Ruby

I've been really enjoying James Edward Gray II's Rubies in the Rough articles. Every couple of weeks, he publishes something that is guaranteed to get me thinking about some aspect of coding I hadn't considered before.

His latest article is part I of an exploration of an algorithm for the Hitting Rock Bottom problem posed by by Gregory Brown & Andrea Singh. At its core, the problem asks you to simulate pouring water into a 2D container, filling it using a simple set of rules.

As I was coding up my solution, I found I had code like

Here " " is a cell containing air, and "~" a watery cell. So clearly we should create some named constants for that:

But it occurred to me that we could use Ruby's singleton methods to give AIR and WATER a little smarts:

Now you could argue that the cave object should do this: cave.watery?, or that the individual elements in the cave should be objects that know their moisture content, rather than simply characters. I don't agree with the first (simply because the cave is the container, and the water/air is the separate stuff that goes into that container). I have a lot of sympathy for the second, and I'd probably end up there given a sufficiently large nudge during a refactoring.

But, for the problem at hand, simply decorating the two constants with a domain method seems to result in code that is a lot more readable. It isn't a technique I'd used before, so I thought I'd share.

(And remember to check out Rubies in the Rough…)

02 Feb 2012 11:30pm GMT

Your customers want rock-solid, bug-free software that does exactly what they expect it to do. Yet they can't always articulate their ideas clearly enough for you to turn them into code. The Cucumber Book dives straight into the core of the problem: communication between people. Cucumber saves the day; it's a testing, communication, and requirements tool - all rolled into one.

31 Jan 2012 9:35pm GMT

Rails 3.2.1 is out, with some fixes and doc improvements. Please check the CHANGELOGs gist for details.

26 Jan 2012 11:33pm GMT

So we didn't quite make the December release date as we intended, but hey, why break a good tradition and start hitting release targets now! In any case, your patience has been worldly rewarded young grasshopper: Rails 3.2 is done, baked, tested, and ready to roll!

I've been running on 3-2-stable for a few months working on Basecamp Next and it's been a real treat. The new faster dev mode in particular is a major step up over 3.1.

Do remember that this is the last intended release series that's going to support Ruby 1.8.7. The master git branch for Rails is now targeting Rails 4.0, which will require Ruby 1.9.3 and above. So now is a great time to start the work on getting your app ready for the current version of Ruby. Let's not hang around old versions forever and a Sunday like those Python guys :).

There's a v3.2.0 tag on Github and we of course we still have the 3-2-stable branch as well. You can see all the glorious details of everything that was changed in our CHANGELOG compilation.

For documentation, we have the 3.2 release notes with upgrade instructions, both the API docs and the guides have been generated for 3.2 as well, and there's a brand new 3.2-compatible version of Agile Web Development with Rails. A smörgåsbord indeed!

Note: If you're having trouble installing the gems under Ruby 1.8.7, you've probably hit a RubyGems bug with YAML that's been fixed in RubyGems 1.8.15. You can upgrade RubyGems using "gem update-system".

If you can't be bothered with the full release notes, here's a reprint of a few feature highlights from when we did the first release candidate:

Faster dev mode & routing

The most noticeable new feature is that development mode got a ton and a half faster. Inspired by Active Reload, we now only reload classes from files you've actually changed. The difference is dramatic on a larger application.

Route recognition also got a bunch faster thanks to the new Journey engine and we made linking much faster as well (especially apparent when you're having 100+ links on a single page).

Explain queries

We've added a quick and easy way to explain quieries generated by ARel. In the console, you can run something like puts Person.active.limit(5).explain and you'll get the query ARel produces explained (so you can easily see whether its using the right indexes). There's even a default threshold in development mode where if a query takes more than half a second to run, it's automatically explained inline-how about that!

Tagged logger

When you're running a multi-user, multi-account application, it's a great help to be able to filter the log by who did what. Enter the TaggedLogging wrapper. It works like this:

Logger = ActiveSupport::TaggedLogging.new(Logger.new(STDOUT))
Logger.tagged("BCX") { Logger.info "Stuff" } # Logs "[BCX] Stuff" 
Logger.tagged("BCX") do
  Logger.tagged("Jason") do
    Logger.info "Stuff" # Logs "\[BCX\] \[Jason\] Stuff" 
  end
end

Active Record Store

Key/value stores are great, but it's not always you want to go the whole honking way just for a little variable-key action. Enter the Active Record Store:

class User < ActiveRecord::Base
  store :settings, accessors: [ :color, :homepage ]
end

u = User.new(color: 'black', homepage: '37signals.com')
u.color                          # Accessor stored attribute
u.settings[:country] = 'Denmark' # Any attribute, even if not specified with an accessor

20 Jan 2012 6:36pm GMT

Dear Sen. Klobuchar, Sen. Franken, and Rep. Ellison,

I am writing to express my deep concern about the pending SOPA/PIPA legislation. I have seen reports that Sen. Franken and Sen. Klobuchar are both in favor of this legislation. This troubles me.

I see the intense legal pressure applied by the movie and recording industries to Congress as evidence that they simply don't understand how to run their businesses in an ever changing technological landscape. They should realize that they could increase revenues by embracing new channels of distribution instead of wasting time tracking lost revenues due to piracy or fraud.

Based on this view, I see SOPA/PIPA as yet another misguided government attempt to intervene, prop up, and rescue an industry that doesn't know how to alter its own business models to align with the current economic and technological climate. This attempt comes at the expense of taxpayers and citizens who view freedom of speech and information as inherent, unalienable rights.

Frankly, I find the amount of money raised for Sen. Franken and Sen. Klobuchar to be a disgusting example of special interests buying legislation. As a constituent of the the state of Minnesota and its fifth congressional district, I urge Sen. Franken, Sen. Klobuchar, and Rep. Ellison to return all funds contributed for and against this legislation and to remove any support for this bill.

Regards,

Nick Sieger

13 Jan 2012 9:00pm GMT

Hi everyone,

Rails 3.2.0.rc2 has been released!

What to update in your apps

• Update your Gemfile to depend on rails ~> 3.2.0.rc2
• Update your Gemfile to depend on sass-rails ~> 3.2.3
• Start moving any remaining Rails 2.3-style vendor/plugins/*. These are finally deprecated!

Extract your vendor/plugins to their own gems and bundle them in your Gemfile. If they're tiny, not worthy of the own gem, fold it into your app as lib/myplugin/* and config/initializers/myplugin.rb.

Changes since RC1

ActionMailer

• No changes

ActionPack

• Add font_path helper method Santiago Pastorino

• Depends on rack ~> 1.4.0 Santiago Pastorino

• Add :gzip option to caches_page. The default option can be configured globally using page_cache_compression Andrey Sitnik

ActiveModel

• No changes

ActiveRecord

• No changes

ActiveResource

• No changes

ActiveSupport

• ActiveSupport::Base64 is deprecated in favor of ::Base64. Sergey Nartimov

Railties

• Rails 2.3-style plugins in vendor/plugins are deprecated and will be removed in Rails 4.0. Move them out of vendor/plugins and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. Santiago Pastorino

• Guides are available as a single .mobi for the Kindle and free Kindle readers apps. Michael Pearson & Xavier Noria

• Allow scaffold/model/migration generators to accept a "index" and "uniq" modifiers, as in: "tracking_id:integer:uniq" in order to generate (unique) indexes. Some types also accept custom options, for instance, you can specify the precision and scale for decimals as "price:decimal{7,2}". Dmitrii Samoilov

Gem checksums

• MD5 (actionmailer-3.2.0.rc2.gem) = 118c83b2cddaa935d1de7534cfb6c810
• MD5 (actionpack-3.2.0.rc2.gem) = 6b18851bc26d5c8958672f27adda05ca
• MD5 (activemodel-3.2.0.rc2.gem) = d82f4eed949dcff17f8bf2aed806679a
• MD5 (activerecord-3.2.0.rc2.gem) = d07806fd5fc464f960200d20ceb2193a
• MD5 (activeresource-3.2.0.rc2.gem) = f51af240ff4623b0b6f8a4293ffa50dc
• MD5 (activesupport-3.2.0.rc2.gem) = 01380240c12e0380c9e61c97dd45f2f1
• MD5 (rails-3.2.0.rc2.gem) = 134f923f7d821f514abf6bdf4af62ca7
• MD5 (railties-3.2.0.rc2.gem) = 4b3ac0f9c5da16b90a1875e8199253d2

You can find an exhaustive list of changes on github. Along with the closed issues marked for v3.2.0.

You can also see issues we haven't closed yet.

Thanks to everyone!

04 Jan 2012 10:12pm GMT

The forthcoming 3.2.x release series will be the last branch of Rails that supports Ruby 1.8.7. There's a new 3-2-stable branch in git to track the changes we need until 3.2.0 final is release and for managing point releases after that.

So for now you should stop floating on rails/master if your application is not compatible with Ruby 1.9.3. We have updated the version numbers to indicate this backwards incompatibility to be 4.0.0.beta. This doesn't mean that 4.0 is anywhere close to being released, mind you. We're simply doing this now because we're dropping support for Ruby 1.8.7 in rails/master and people should know what's up.

Major versions of Rails has been on about 2-year release cycle since 1.0 (released in 2005, followed by 2.0 in 2007, followed by 3.0 in 2010) and we intend to continue this pattern. The current internal target for Rails 4.0 is sometime in the Summer of 2012 - but we have blown every major release estimate in the past, so don't bet your farm on it.

There's not a lot of details about what we're going to include in Rails 4.0 yet as the primary purpose for bumping the major version number is to drop Ruby 1.8.7 support. But unlike Rails 3.0, we intend for it to be a much smoother transition. The intention is not for this to be a REWRITE EVERYTHING release in the same way 3.0 was to some extent.

But we're getting ahead of ourselves. First mission is to get Rails 3.2 out!

20 Dec 2011 4:06pm GMT

Once you've boarded the Rails train, you just know that every stop along the way is going to be a good time. This release candidate is no different and we've packed it with loving goodies without making upgrading a hassle.

Faster dev mode & routing

The most noticeable new feature is that development mode got a ton and a half faster. Inspired by Active Reload, we now only reload classes from files you've actually changed. The difference is dramatic on a larger application.

Route recognition also got a bunch faster thanks to the new Journey engine and we made linking much faster as well (especially apparent when you're having 100+ links on a single page).

Explain queries

We've added a quick and easy way to explain quieries generated by ARel. In the console, you can run something like puts Person.active.limit(5).explain and you'll get the query ARel produces explained (so you can easily see whether its using the right indexes). There's even a default threshold in development mode where if a query takes more than half a second to run, it's automatically explained inline-how about that!

Tagged logger

When you're running a multi-user, multi-account application, it's a great help to be able to filter the log by who did what. Enter the TaggedLogging wrapper. It works like this:

Logger = ActiveSupport::TaggedLogging.new(Logger.new(STDOUT))
Logger.tagged("BCX") { Logger.info "Stuff" } # Logs "[BCX] Stuff" 
Logger.tagged("BCX") do
  Logger.tagged("Jason") do
    Logger.info "Stuff" # Logs "\[BCX\] \[Jason\] Stuff" 
  end
end

Active Record Store

Key/value stores are great, but it's not always you want to go the whole honking way just for a little variable-key action. Enter the Active Record Store:

class User < ActiveRecord::Base
  store :settings, accessors: [ :color, :homepage ]
end

u = User.new(color: 'black', homepage: '37signals.com')
u.color                          # Accessor stored attribute
u.settings[:country] = 'Denmark' # Any attribute, even if not specified with an accessor

These are just a few of the highlights. The full release notes detail every loving change.

Given that this is a release candidate, we're ever so eager to hear your feedback. We hope it'll be a quick RC phase, but please do spoil that plan by reporting bugs.

As always, you install a release candidate by doing gem install rails --pre.

20 Dec 2011 12:48am GMT

There are some new features related to EXPLAIN in the forthcoming Ruby on Rails 3.2 we'd like to share:

• Running EXPLAIN manually
• Automatic EXPLAIN for slow queries
• Silencing automatic EXPLAIN

As of this writing they are available for the adapters sqlite3, mysql2, and postgresql.

Running EXPLAIN Manually

You can now run EXPLAIN on the SQL generated by a relation this way:

User.where(:id => 1).joins(:posts).explain

The result of that method call is a string that carefully imitates the output of database shells. For example, under MySQL you get something similar to

EXPLAIN for: SELECT `users`.* FROM `users` INNER JOIN `posts` ON `posts`.`user_id` = `users`.`id` WHERE `users`.`id` = 1
+----+-------------+-------+-------+---------------+---------+---------+-------+------+-------------+
| id | select_type | table | type  | possible_keys | key     | key_len | ref   | rows | Extra       |
+----+-------------+-------+-------+---------------+---------+---------+-------+------+-------------+
|  1 | SIMPLE      | users | const | PRIMARY       | PRIMARY | 4       | const |    1 |             |
|  1 | SIMPLE      | posts | ALL   | NULL          | NULL    | NULL    | NULL  |    1 | Using where |
+----+-------------+-------+-------+---------------+---------+---------+-------+------+-------------+
2 rows in set (0.00 sec)

and under PostgreSQL the same call yields something like

EXPLAIN for: SELECT "users".* FROM "users" INNER JOIN "posts" ON "posts"."user_id" = "users"."id" WHERE "users"."id" = 1
                                  QUERY PLAN
------------------------------------------------------------------------------
 Nested Loop Left Join  (cost=0.00..37.24 rows=8 width=0)
   Join Filter: (posts.user_id = users.id)
   ->  Index Scan using users_pkey on users  (cost=0.00..8.27 rows=1 width=4)
         Index Cond: (id = 1)
   ->  Seq Scan on posts  (cost=0.00..28.88 rows=8 width=4)
         Filter: (posts.user_id = 1)
(6 rows)

Please note that explain runs the query or queries and asks the database for their respective query plan afterwards. This is because due to eager loading a relation may trigger several queries to fetch the records and their associations, and in such cases some queries need the result of the previous ones.

If the relation triggers several queries the method still returns a single string with all the query plans. This is an output meant for human consumption so we preferred to present everything as a string in a format which is familiar right away rather than a structure.

Automatic EXPLAIN For Slow Queries

Rails 3.2 has the ability to help in detecting slow queries.

New applications get

config.active_record.auto_explain_threshold_in_seconds = 0.5

in config/environments/development.rb. Active Record monitors queries and if they take more than that threshold their query plan will be logged using warn.

That works for anything running find_by_sql (which is almost everything, since most of Active Record ends up calling that method). In the particular case of relations, the threshold is compared against the total time needed to fetch the records, not against the time taken by each individual query. Because conceptually we are concerned with the cost of the call

User.where(:id => 1).joins(:posts).explain

rather than the cost of the different queries that call may trigger due to the implementation.

By default the threshold is nil in the test and production environments, which means the feature is disabled.

The value of that parameter is nil also if the threshold is not set, so existing applications will need to add it by hand if they migrate to 3.2 to be able to enable automatic EXPLAIN.

Silencing Automatic EXPLAIN

With automatic EXPLAIN enabled, it could still be the case that some queries are just slow and you know they have to be. For example, a heavyweight report in the backoffice.

The macro silence_auto_explain allows you to avoid having EXPLAIN run repeatedly in those areas of code:

ActiveRecord::Base.silence_auto_explain do
  # no automatic EXPLAIN here
end

Interpreting Query Plans

The interpretation of the query plans is another topic, these are some pointers:

• SQLite: EXPLAIN QUERY PLAN
• MySQL: EXPLAIN Output Format
• PostgreSQL: Using EXPLAIN

Happy debugging!

07 Dec 2011 11:23am GMT

Take advantage of Sinatra, the Ruby-based web application library and domain-specific language used by GitHub, LinkedIn, Engine Yard, and other prominent organizations. With this concise book, you will quickly gain working knowledge of Sinatra and its minimalist approach to building both standalone and modular web applications.

24 Nov 2011 8:33pm GMT

Rails 3.1.3 has been released. This release mainly contains fixes for regressions that popped up in 3.1.2.

Changes

Action Pack:

• Downgrade sprockets to ~> 2.0.3. Using 2.1.0 caused regressions.

• Fix using translate helper with a html translation which uses the :count option for pluralization.

  Jon Leighton

Active Record:

• Perf fix: If we're deleting all records in an association, don't add a IN(..) clause to the query. GH 3672

  Jon Leighton

• Fix bug with referencing other mysql databases in settablename. GH 3690

• Fix performance bug with mysql databases on a server with lots of other databses. GH 3678

  Christos Zisopoulos and Kenny J

Railties:

• New apps should be generated with a sass-rails dependency of 3.1.5, not 3.1.5.rc.2

As ever, you can see a full list of changes on Github.

20 Nov 2011 11:14pm GMT

Rails 3.1.2 has been released. This is a patch-level release containing bug fixes and an important security fix.

Possible XSS vulnerability in the translate helper method in Ruby on Rails

There is a vulnerability in the translate helper method which may allow an attacker to insert arbitrary code into a page.

• Versions Affected: 3.0.0 and later, 2.3.X in combination with the rails_xss plugin
• Not Affected: Pre-3.0.0 releases, without the rails_xss plugin, did no automatic XSS escaping, so are not considered vulnerable
• Fixed Versions: 3.0.11, 3.1.2

Please see the rubyonrails-security posting and the changelog item below, for more details.

Changes

Action Mailer:

• No changes

Action Pack:

• Fix XSS security vulnerability in the translate helper method. When using interpolation in combination with HTML-safe translations, the interpolated input would not get HTML escaped. GH 3664

  Before:

  translate('foo_html', :something => '<script>') # => "...<script>..."

  After:

  translate('foo_html', :something => '<script>') # => "...&lt;script&gt;..."

  Sergey Nartimov

• Upgrade sprockets dependency to ~> 2.1.0

• Ensure that the format isn't applied twice to the cache key, else it becomes impossible to target with expire_action.

  Christopher Meiklejohn

• Swallow error when can't unmarshall object from session.

  Bruno Zanchet

• Implement a workaround for a bug in ruby-1.9.3p0 where an error would be raised while attempting to convert a template from one encoding to another.

  Please see http://redmine.ruby-lang.org/issues/5564 for details of the bug.

  The workaround is to load all conversions into memory ahead of time, and will only happen if the ruby version is exactly 1.9.3p0. The hope is obviously that the underlying problem will be resolved in the next patchlevel release of 1.9.3.

  Jon Leighton

• Ensure users upgrading from 3.0.x to 3.1.x will properly upgrade their flash object in session (issues #3298 and #2509)

Active Model:

• No changes

Active Record:

• Fix problem with prepared statements and PostgreSQL when multiple schemas are used. GH #3232

  Juan M. Cuello

• Fix bug with PostgreSQLAdapter#indexes. When the search path has multiple schemas, spaces were not being stripped from the schema names after the first.

  Sean Kirby

• Preserve SELECT columns on the COUNT for finder_sql when possible. GH 3503

  Justin Mazzi

• Reset prepared statement cache when schema changes impact statement results. GH 3335

  Aaron Patterson

• Postgres: Do not attempt to deallocate a statement if the connection is no longer active.

  Ian Leitch

• Prevent QueryCache leaking database connections. GH 3243

  Mark J. Titorenko

• Fix bug where building the conditions of a nested through association could potentially modify the conditions of the through and/or source association. If you have experienced bugs with conditions appearing in the wrong queries when using nested through associations, this probably solves your problems. GH #3271

  Jon Leighton

• If a record is removed from a has_many :through, all of the join records relating to that record should also be removed from the through association's target.

  Jon Leighton

• Fix adding multiple instances of the same record to a has_many :through. GH #3425

  Jon Leighton

• Fix creating records in a through association with a polymorphic source type. GH #3247

  Jon Leighton

• MySQL: use the information_schema than the describe command when we look for a primary key. GH #3440

  Kenny J

Active Resource:

• No changes

Active Support:

• No changes

Railties:

• Engines: don't blow up if db/seeds.rb is missing.

  Jeremy Kemper

• rails new foo --skip-test-unit should not add the :test task to the rake default task. GH 2564

  José Valim

As ever, you can see a full list of commits between the versions on Github.

18 Nov 2011 1:57am GMT

Rails 3.0.11 has been released. This is a patch-level release containing bug fixes and an important security fix.

Possible XSS vulnerability in the translate helper method in Ruby on Rails

There is a vulnerability in the translate helper method which may allow an attacker to insert arbitrary code into a page.

• Versions Affected: 3.0.0 and later, 2.3.X in combination with the rails_xss plugin
• Not Affected: Pre-3.0.0 releases, without the rails_xss plugin, did no automatic XSS escaping, so are not considered vulnerable
• Fixed Versions: 3.0.11, 3.1.2

Please see the rubyonrails-security posting and the changelog item below, for more details.

Changes

Action Mailer:

• No changes

Action Pack:

• Fix XSS security vulnerability in the translate helper method. When using interpolation in combination with HTML-safe translations, the interpolated input would not get HTML escaped. GH 3664

  Before:

  translate('foo_html', :something => '<script>') # => "...<script>..."

  After:

  translate('foo_html', :something => '<script>') # => "...&lt;script&gt;..."

  Sergey Nartimov

• Implement a workaround for a bug in ruby-1.9.3p0 where an error would be raised while attempting to convert a template from one encoding to another.

  Please see http://redmine.ruby-lang.org/issues/5564 for details of the bug.

  The workaround is to load all conversions into memory ahead of time, and will only happen if the ruby version is exactly 1.9.3p0. The hope is obviously that the underlying problem will be resolved in the next patchlevel release of 1.9.3.

• Fix assert_select_email to work on multipart and non-multipart emails as the method stopped working correctly in Rails 3.x due to changes in the new mail gem.

• Fix url_for when passed a hash to prevent additional options (eg. :host, :protocol) from being added to the hash after calling it.

Active Model:

• No changes

Active Record:

• Exceptions from database adapters should not lose their backtrace.

• Backport "ActiveRecord::Persistence#touch should not use default_scope" (GH #1519)

• Psych errors with poor yaml formatting are proxied. Fixes GH #2645 and GH #2731

• Fix ActiveRecord#exists? when passsed a nil value

Active Resource:

• No changes

Active Support:

• No changes

Railties:

• Updated Prototype UJS to lastest version fixing multiples errors in IE [Guillermo Iguaran]

As ever, you can see a full list of commits between the versions on Github.

18 Nov 2011 1:55am GMT

So the Hacker News folks are having a discussion about my email experiment last April. Many interesting points were raised. One fair question is "how did the experiment work out?"

In short, it worked incredibly well.

I was mostly worried about annoying the folks sending me email. But the only feedback I got was positive.

I was also a little worried about folks abusing the [urgent] flag. But that didn't happen, either. I had perhaps 5 or 6 urgent emails, and they were indeed things I needed to handle when I got back. Maybe I'm just lucky when it comes to the people who correspond with me.

The experiement had two positive effects on my life. First, the vacation was genuinely a lot nicer not having to worry about the sacks full of mail piling up for me when I got back. Was that selfish of me? Perhaps a little.

I wasn't expecting the other side effect. Since I returned from vacation, the quality of email I receive has improved, and the quantity I receive has dropped. I still enjoy interacting with all the people I need to interact with, and I still get to answer all the questions that need answering. It just seems that my inbox is somehow more focussed.

I have a theory. I think that, during the course of the preceding few years, I'd become something of a slave to my email. I'd answer stuff as it arrived. And those rapid responses would in turn trigger another round of email, and another. There was almost an adreneline rush to it.

So my vacation broke that cycle. And now things are sane (or at least closer to sane).

13 Nov 2011 4:29pm GMT

Ruby and Cocoa on OS X, the iPhone, and the Device That Shall Not Be Named

28 Oct 2011 8:43pm GMT

This was originally posted as a comment on @fogus's blog post "Why Clojure doesn't need invokedynamic, but it might be nice". I figured it's worth a top-level post here.

Ok, there's some good points here and a few misguided/misinformed positions. I'll try to cover everything.

First, I need to point out a key detail of invokedynamic that may have escaped notice: any case where you must bounce through a generic piece of code to do dispatch -- regardless of how fast that bounce may be -- prevents a whole slew of optimizations from happening. This might affect Java dispatch, if there's any argument-twiddling logic shared between call sites. It would definitely affect multimethods, which are using a hand-implemented PIC. Any case where there's intervening code between the call site and the target would benefit from invokedynamic, since invokedynamic could be used to plumb that logic and let it inline straight through. This is, indeed, the primary benefit of using invokedynamic: arbitrarily complex dispatch logic folds away allowing the dispatch to optimize as if it were direct.

Your point about inference in Java dispatch is a fair one...if Clojure is able to infer all cases, then there's no need to use invokedynamic at all. But unless Clojure is able to infer all cases, then you've got this little performance time bomb just waiting to happen. Tweak some code path and obscure the inference, and kablam, you're back on a slow reflective impl. Invokedynamic would provide a measure of consistency; the only unforeseen perf impact would be when the dispatch turns out to *actually* be polymorphic, in which case even a direct call wouldn't do much better.

For multimethods, the benefit should be clear: the MM selection logic would be mostly implemented using method handles and "leaf" logic, allowing hotspot to inline it everywhere it is used. That means for small-morphic MM call sites, all targets could potentially inline too. That's impossible without invokedynamic unless you generate every MM path immediately around the eventual call.

Now, on to defs and Var lookup. Depending on the cost of Var lookup, using a SwitchPoint-based invalidation plus invokedynamic could be a big win. In Java 7u2, SwitchPoint-based invalidation is essentially free until invalidated, and as you point out that's a rare case. There would essentially be *no* cost in indirecting through a var until that var changes...and then it would settle back into no cost until it changes again. Frequently-changing vars could gracefully degrade to a PIC.

It's also dangerous to understate the impact code size has on JVM optimization. The usual recommendation on the JVM is to move code into many small methods, possibly using call-through logic as in multimethods to reuse the same logic in many places. As I've mentioned, that defeats many optimizations, so the next approach is often to hand-inline logic everywhere it's used, to let the JVM have a more optimizable view of the system. But now we're stepping on our own feet...by adding more bytecode, we're almost certainly impacting the JVM's optimization and inlining budgets.

OpenJDK (and probably the other VMs too) has various limits on how far it will go to optimize code. A large number of these limits are based on the bytecoded size of the target methods. Methods that get too big won't inline, and sometimes won't compile. Methods that inline a lot of code might not get inlined into other methods. Methods that inline one path and eat up too much budget might push out more important calls later on. The only way around this is to reduce bytecode size, which is where invokedynamic comes in.

As of OpenJDK 7u2, MethodHandle logic is not included when calculating inlining budgets. In other words, if you push all the Java dispatch logic or multimethod dispatch logic or var lookup into mostly MethodHandles, you're getting that logic *for free*. That has had a tremendous impact on JRuby performance; I had previous versions of our compiler that did indeed infer static target methods from the interpreter, but they were often *slower* than call site caching solely because the code was considerably larger. With invokedynamic, a call is a call is a call, and the intervening plumbing is not counted against you.

Now, what about negative impacts to Clojure itself...

#0 is a red herring. JRuby supports Java 5, 6, and 7 with only a few hundred lines of changes in the compiler. Basically, the compiler has abstract interfaces for doing things like constant lookup, literal loading, and dispatch that we simply reimplement to use invokedynamic (extending the old non-indy logic for non-indified paths). In order to compile our uses of invokedynamic, we use Rémi Forax's JSR-292 backport, which includes a "mock" jar with all the invokedynamic APIs stubbed out. In our release, we just leave that library out, reflectively load the invokedynamic-based compiler impls, and we're off to the races.

#1 would be fair if the Oracle Java 7u2 early-access drops did not already include the optimizations that gave JRuby those awesome numbers. The biggest of those optimizations was making SwitchPoint free, but also important are the inlining discounting and MutableCallSite improvements. The perf you see for JRuby there can apply to any indirected behavior in Clojure, with the same perf benefits as of 7u2.

For #2, to address the apparent vagueness in my blog post...the big perf gain was largely from using SwitchPoint to invalidate constants rather than pinging a global serial number. Again, indirection folds away if you can shove it into MethodHandles. And it's pretty easy to do it.

#3 is just plain FUD. Oracle has committed to making invokedynamic work well for Java too. The current thinking is that "lambda", the support for closures in Java 7, will use invokedynamic under the covers to implement "function-like" constructs. Oracle has also committed to Nashorn, a fully invokedynamic-based JavaScript implementation, which has many of the same challenges as languages like Ruby or Python. I talked with Adam Messinger at Oracle, who explained to me that Oracle chose JavaScript in part because it's so far away from Java...as I put it (and he agreed) it's going to "keep Oracle honest" about optimizing for non-Java languages. Invokedynamic is driving the future of the JVM, and Oracle knows it all too well.

As for #4...well, all good things take a little effort :) I think the effort required is far lower than you suspect, though.

14 Oct 2011 2:40pm GMT