The main interface method in an OutputFormat is:

  public void format(OutputStream o, Event e) throws IOException;

Now, the reality of the way Flume uses this interface is that it repeatedly calls the method on the same OutputStream with the same OutputFormat. For any period of time, the OutputFormat is only used with one OutputStream, and visa versa. For the lifetime of the OutputStream, it is only used with one OutputFormat object.

So you might think that you could buffer some output inside the OutputFormat. The problem here is that we only partially understood the usage of the OutputFormat interface in Flume.

In particular, Flume has an end-to-end reliability mode, which needs to know for sure if an Event has been written to a destination. If your OutputFormat buffers any data, then Flume has no way of knowing whether a particular Event got written to an OutputStream.

This means that any compression must be at best a per-record compression – the compression Codec must not have any buffered unwritten data when format() returns.

Interestingly, this was implicit in the interface, on some level – the fact that the format() method takes an OutputStream parameter would implicitly seem to imply that you shouldn’t buffer data. But the implementations generally assume the usage pattern above, keeping a reference to the current OutputStream for the duration. They just also did an internal “flush()” call which we tried to remove in our implementation.

See: https://issues.apache.org/jira/browse/FLUME-983

If I go to the IMDB site, or use the iPad or Android app, a search for the episode title fails. I have to search for the show, navigate to the correct season (which is a guess, because I usually only know the year of broadcast,) and then browse the list to find the episode.

The workaround to this is to Google it. Add “site:imdb.com” to your Google search, as well as episode title and series name.

Another deficiency of the IMDB apps (but not the web interface,) is that if I find an actor, and his credits include, say, an episode of House, selecting that credit doesn’t take me to the episode, but to the House front page. That is hardly ever what I want.

This is bad, because I see the word “ON,” which is in all-caps, and I think that “Silent Mode” is “ON,” when that is not what it means at all. The header and the secondary line are contradictory, leading to confusion. (Yes, it turns out that if the phone is actually in “Silent Mode,” the icon changes, but that’s not obvious to the new user, and it’s very easy to make this mistake.)

Also, why are “Silent” and “Airport” modes listed as “ON” or “OFF,” but “Data Network Mode” is “activated?” (I suspect the reason for this that Data Network Mode” costs the user money, but it’s not clear why the text is lower case, when “ON” and “OFF” are upper case.