daily snapshot | 10 february 2011

February 10, 2011
  • Reverse incentive – bootstrap the writing by introducing “motivational” example as core problem
  • Cloudera | Flume – handling thrift dependencies -> we should be able to build without system-level Thrift support
  • Troubles with ANTLR runtime and interpreting in-line Flume query syntax – even a basic syntax errors result in unreadable exceptions – something to work on
  • Stream splitting using arbitrary keys – “signaled” in Decorator-time with sink writing data to multiple outputs
  • Millennial generations used to ever-changing reality began to sense the world – transient, unstable, capable of change in the tune of their ideas. For the first time – no big questions – no irony in existence. Just like science a century ago – technology brought in a new religion – belief in the potential for change of the very fiber of things – controllably manipulating senses, twisting the data, creating new worlds. Eternal contempt for the flesh, once again sensed a way out. It is clearly visible how the media of the time had a genuine belief that they have made it this time  – that the filters and colors put in place to create these synthetic realities are finally making us free – that we have won. So what happened ? Why, only a decade later, we’re back waging the old war, touching decaying bodies, sensing the aging skin and learning to walk again. Why is it that we’re once again trembling in fear, thankful for every fragment of sensation we can get our hands on. And once again dancing in oblivion, waiting for new hope to arrive ?
  • flume node_nowatch -n dump -c “node:src|snk;”
  • Adding new Flume decorators ? Standard ones are defined on reserved-keyword level.
  • Adding new data sources ? SourceFactoryImpl holds a static list of source->class mappings – not available on the conf-level
  • Establishing simple host-to-host console pipe:

host01$ flume node_nowatch -1 -s -n dump -c ‘dump: console | agentBESink(“host02″);’
host02$ flume node_nowatch -1 -s -n dump -c ‘dump: collectorSource(35853) | console;’


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: