Skip to content

Fluentd output plugin which can write to PostgreSQL hstore extension.

License

Notifications You must be signed in to change notification settings

shirou/fluent-plugin-pghstore

Repository files navigation

fluent-plugin-pghstore

Component

PgHStoreOutput

Output to PostgreSQL hstore database.

Output table should have tag, time and record column.:

CREATE TABLE #{tablename} (
  tag TEXT[],
  time TIMESTAMP WITH TIME ZONE,
  record HSTORE
);

A tag is splited by “.” and stored in the TEXT[]. (Yes, postgres can handle an array!)

Requirement

  • PostgreSQL 9.0 or higher

    • hstore changed at PostgreSQL 9.0.

  • postgres-contrib

  • ruby-pg

How to Install hstore

hstore is in the contrib.

9.1 or higher:

psql <dbname> -c "CREATE EXTENSION hstore;"

9.0:

psql <dbname> -f ${PGHOME}/share/contrib/hstore.sql

Configuration

Example:

<match apache.*>
  type pghstore
  database test
  table test
  table_option CREATE INDEX time_index ON testb (time);
</match>

Options

  • Required

    • database

      • database name

  • Optional

    • table

      • tablename.

      • If not set, use fluentd_store.

      • If not exists, creates automatically.

    • host

    • port

    • user

    • password

    • table_option

      • Add some SQL. This SQL is called only once when table is created from this plugin.

Limitation

  • Nested output is not allowd.

  • Since using only one connection, the performance may become bad. When you meet this, use connection pooling and write patch!

Thanks

This source code is mainly borrowed from

{fluent-plugin-datacounter}{rubygems.org/gems/fluent-plugin-datacounter}. Thank you for tagomoris.

Copyright

Copyright © 2012- Shirou WAKAYAMA

License

Apache License

ChangeLog

  • 0.2.2 / 2013-04-20

0.2.1 change caused error in 9.2. 0.2.1 change was not necessary. Thank you kaz29!

  • 0.2.1 / 2012-09-11

PostgreSQL 9.2 {removes hstore’s => operator}{wiki.postgresql.org/wiki/What’s_new_in_PostgreSQL_9.2#Remove_hstore.27s_.3D.3E_operator}. This plugin checks the server’s version and change the operator.

But not fully tested yet.

About

Fluentd output plugin which can write to PostgreSQL hstore extension.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages