Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark 0 #4

Merged
merged 23 commits into from
Nov 16, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b51e2db
:snowflake: basic setup #3
jrans Oct 25, 2016
88b5368
:heavy_plus_sign: port over code from dwyl/abase #2
jrans Oct 25, 2016
bb889a4
:heavy_plus_sign: :art: :white_check_mark: Clean up past code #2
jrans Oct 25, 2016
1eeacd1
:heavy_plus_sign: Except single schema or array #5
jrans Oct 26, 2016
15ad9cc
:heavy+plus_sign: Expose db handlers from module #7
jrans Oct 26, 2016
e9b9d55
:heavy_plus_sign: FLush handler #9
jrans Oct 27, 2016
6785621
:bug: make sure right db conenction being passed
jrans Oct 28, 2016
84d8bda
:bug: to tables with same unqiue field clash fix
jrans Oct 30, 2016
44c6ef8
:heavy_plus_sign: expose validate and createClient helpers #10 #11
jrans Oct 30, 2016
1b1308b
0.2.0
jrans Oct 31, 2016
89316b1
:memo: Add example to repo #6
jrans Nov 1, 2016
65f98c9
:green_heart: Add CI #3
jrans Nov 1, 2016
392b943
:memo: Add complete documentation for work so far #8
jrans Nov 1, 2016
75f9988
:green_heart: add database setup to travis #3
jrans Nov 1, 2016
f18c787
:heavy_plus_sign: type id #12
jrans Nov 15, 2016
6b69a13
:heavy_plus_sign: primary key constraint #12
jrans Nov 15, 2016
028eb60
:memo: Readme update for id and primarykey #12
jrans Nov 15, 2016
e659b26
0.2.1
jrans Nov 15, 2016
4bc965a
:bug: ensure blank arg passed to aguid to guarentee uniquness
jrans Nov 15, 2016
f00114f
0.2.2
jrans Nov 15, 2016
8739d16
:heavy_plus_sign: Return generated ids in the response #12
jrans Nov 15, 2016
e35f05f
0.2.3
jrans Nov 15, 2016
bfbad44
Merge pull request #14 from dwyl/id-generation
nelsonic Nov 15, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
root = true

[*]
indent_style = space
indent_size = 2
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
1 change: 1 addition & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
coverage
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,9 @@ jspm_packages

# Optional REPL history
.node_repl_history

# goodparts symlink
.eslintrc.js

# environment variables
*.env
12 changes: 12 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
language: node_js
node_js:
- "4"
- "6"
services:
- postgresql
before_script:
- psql -c 'create database testdb;' -U postgres
before_install:
- pip install --user codecov
after_success:
- codecov --file coverage/lcov.info --disable search
190 changes: 190 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,192 @@
# joi-postgresql
A little experiment in defining models in Joi and creating PostgreSQL Tables

[![Build Status](https://travis-ci.org/dwyl/joi-postgresql.svg?branch=master)](https://travis-ci.org/dwyl/joi-postgresql)
[![codecov](https://codecov.io/gh/dwyl/joi-postgresql/branch/master/graph/badge.svg)](https://codecov.io/gh/dwyl/joi-postgresql)
[![Code Climate](https://codeclimate.com/github/dwyl/joi-postgresql/badges/gpa.svg)](https://codeclimate.com/github/dwyl/joi-postgresql)
[![dependencies Status](https://david-dm.org/dwyl/joi-postgresql/status.svg)](https://david-dm.org/dwyl/joi-postgresql)
[![devDependencies Status](https://david-dm.org/dwyl/joi-postgresql/dev-status.svg)](https://david-dm.org/dwyl/joi-postgresql?type=dev)
[![contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://github.com/dwyl/joi-postgresql/issues)

## abase-db

### What?
abase-db is a [hapi](https://github.com/hapijs/hapi) plugin that provides an easy way to set up postgres database tables and perform CRUD operations by declaring a schema object which is heavily influenced by [joi](https://github.com/hapijs/joi).

It can be used alone but is most powerful when used as part of [abase](https://github.com/dwyl/abase) or with your select few abase plugins.

Note if you are totally new to Hapi.js see: https://github.com/dwyl/learn-hapi
And/or if you are new to postgres check out: https://github.com/dwyl/learn-postgresql

### Why?

From a joi schema we should be able to infer many things about the fields in a database. `abase-db` provides the mapping between a config (inspired by joi schema) to commands that can create tables with the correct fields.

We also want a "plug and play" access and easy to use handlers to perform CRUD operations and `abase-db` offers this without having to worry about any postgres querying.

For more understanding of *why* see the parent module [abase]((https://github.com/dwyl/abase)) as this provides just the db part.

> #### Why PostgreSQL?

> While there is a lot of hype surrounding NoSQL Databases like MongoDB & Redis, we found we were having to write a lot of code to do useful queries. And while de-normalising data might "make sense" for "scalability" in theory, what we found in practice is that even with 100 Billion Records (way more users than 99.99% of companies/startups!) a well-managed PostgreSQL cluster copes very well.

> Make up your own mind: https://www.postgresql.org/about
If you're still Curious or Worried about scaling PostgreSQL? see: https://www.citusdata.com Want to model the network of people as a graph? https://github.com/cayleygraph/cayley

### How?

1. Install `npm install abase-db --save`
2. Write a schema for your tables like so:
```js
var schema = {
tableName: 'users',
fields: {
name: { type: 'string' }
}
}
```
3. Run a database remotely or locally (see [here](https://github.com/dwyl/learn-postgresql) for how) and acquire db url or connection object.
4. Create options object of the form:
```js
var options = {
dbConnection: process.env.DATABASE_URL,
schema: dbSchema
};
```
5. Plugin
```js
server.register([
{ register: require('abase-db'), options: options }
], function () {
server.route(routes);
server.start();
});
```
6. Play
```js
handler: function (request, reply) {
return request.abase.db.insert(
{ tableName: 'users', fields: request.payload },
function () { return reply('OK') }
);
}
```
7. Play without hapi. See API section below.

### API

#### Plugin: `require('abase-db')`

##### Registration
When registered with Hapi takes options of the form:
```
{ dbConnection, schema }
```
###### dbConnection
Either provide a database url and we'll do the rest or an object that can used to configure a pooled connection with [node-pg](https://github.com/brianc/node-postgres#client-pooling).
###### Schema

The schema is in align with the requirements made by [abase]((https://github.com/dwyl/abase)) and as stated before is inspired by joi and will try to provide a one to one mapping.

The schema must be an object (or an array of objects for multiple tables) of the form: `{ tableName, fields }`.

`fields` is of the form `{ [fieldName]: { type, rest: optional }`

Table and field names must be valid postgres table and column names. (non empty, alphanumeric, no leading number, less than 64)

Each field must be given a type prop. Data/joi types we support:

| Joi type (type prop for field)| Postgres type | Notes |
|---|---|---|
| `date` | `DATE` or `TIMESTAMP` | set `timestamp: true` for latter |
| `number` | `DOUBLE PRECISION` or `BIGINT` | set `integer: true` for latter |
| `string` | `VARCHAR(80 or max)` | `80` default, set `max: 123` as you like for more/less |
|boolean | BOOLEAN | |
| `id` | VARCHAR(36) | **warning** if using this type do not add this field to your insert, we will generate an id on each insertion (Generated with [aguid](https://github.com/dwyl/aguid)) |

More information can be inferred from `lib/config_validator.js`

Each field can also take more properties most of which will be used by other abase modules and have no effect but the ones we care about right now are.

| Property | Notes |
|---|---|
| `unique` | set to `true` if you want column unique |
| `primaryKey` | set to `true` if you want this field to act as your primary key (note only one field allowed!) |
| `max`, `timestamp`, `integer` | see types table above for relevance |

##### Under the hood

###### Table Set Up
With given database and schema, on initialisation of plugin, we will create all necessary tables if they don't already exist.

This will only therefore happen if starting server for the first time, or if a new table is added to the schema.

**Unfortunately** if you want to modify a tables schema you will have to drop the whole table to have the database reconfigured on start up. We look to find a nice process for this in the future if you want to update your tables with new columns.

###### Request decoration

Each request will have the db handlers `insert`, `select`, `update`, `delete`. They all have clients attached and ready to go.

They can be accessed like so: `request.abase.db.insert`.

They are all of the form `function(options, callback = optional)` and return promises if no callback given.

The `options` object must contain `tableName`, i.e. the table you want to operate on. Below are more details for properties of options.

| Property | Used in | Notes |
| --- | --- | --- |
| `fields` | `insert`, `update` | Object with field names and values corresponding to the schema provided |
| `select` | `select` | array of keys which want to be retrieved, if not present defaults to all columns |
| `where` | `select`, `update`, `delete` | object with field names and values that must match by equality (would like inequality in future) |

###### Server decoration

The hapi server will be given a method `endAbaseDb` of the form `function (callback)` which can be called to closed the pool connection.

##### use

#### validate: `require('abase-db').validate`

Helper that you can use to check your schema outside of hapi. Takes in a schema object and will throw if it fails.

#### createConnection: `require('abase').createConnection`

Helper of the form `function(dbConnection)` to create a single node-pg client that is configured in the same way as how you provide your dbConnection above.

#### handlers: `require('abase').handlers`

Object with methods `insert`, `select`, `update`, `delete`, `init`, `flush`.

They all have form `function(client, schema, options, cb)` so you will have to bind your own client.

Crud operation documented above.

##### init
Used at plugin registration takes same schema and doesn't use options arg.

##### flush
Used to drop tables easily. If given options arg will delete on a table by table basis but if left out will delete all tables in schema.
options takes the form `{tableName}`.

### Examples and tests

#### setup

For examples and tests you will need a `config.env` file at the root of your project with a test database url like so:
```
TEST_DATABASE_URL=psql://localhost:5432/testdb
```

Note: this database must be running and before tests are run the tables may be removed from the database so don't keep anything important there.

#### Simple example

To see a simple example in action type `npm run example` into your command line.

### Questions and Suggestions

We hope you find this module useful!

If you need something cleared up, have any requests or want to offer any improvements then please create an issue or better yet a PR!

Note We are aware that not all postgres features may be supported yet. This module will need a few iterations so please suggest missing features to be implemented as you use it and we can hopefully work together to solve it.
34 changes: 34 additions & 0 deletions example/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
'use strict';

var env = require('env2')('config.env'); // eslint-disable-line

var Hapi = require('hapi');
var hoek = require('hoek');
var AbaseDb = require('../lib/');
var routes = require('./routes.js');
var dbSchema = require('./schema.js');

var server = new Hapi.Server();

var abaseDbOptions = {
dbConnection: process.env.TEST_DATABASE_URL,
schema: dbSchema
};

server.connection({ port: 8000 });

server.register([
{ register: AbaseDb, options: abaseDbOptions }
], function (err) {
hoek.assert(!err, err);

server.route(routes);

server.start(function (error) {
hoek.assert(!error, error);

console.log('Visit: http://localhost:' + server.info.port + '/'); // eslint-disable-line
});
});

module.exports = server;
72 changes: 72 additions & 0 deletions example/routes.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
'use strict';

var newPost = '<form action="/new" method="POST" >'
+ '<input name = "title" >'
+ '<input name = "body" >'
+ '<input type="submit" value="Create" />'
+ '</form>'
;

function existingPost (post) {
var id = post.id;
var title = post.title;
var body = post.body;

return '<div><form action="/update/ ' + id + '" method="POST" >'
+ '<input name = "title" value = "' + title + '">'
+ '<input name = "body" value = "' + body + '">'
+ '<input type="submit" value="Update" />'
+ '</form>'
+ '<form action="/delete/' + id + '" method="GET" >'
+ '<input type="submit" value="Delete" />'
+ '</form></div>'
;
}

module.exports = [{
method: 'GET',
path: '/',
handler: function (request, reply) {
return request.abase.db.select({ tableName: 'posts' }, function (_, data) {
var sortedRows = data.rows.sort(function (a, b) {
return a.id > b.id;
});

return reply(newPost + sortedRows.map(existingPost).join('<br/>'));
});
}
}, {
method: 'POST',
path: '/new',
handler: function (request, reply) {
var id = Date.now();
var fields = Object.assign({ id: id }, request.payload);

return request.abase.db.insert(
{ tableName: 'posts', fields: fields },
function () { return reply.redirect('/') }
);
}
}, {
method: 'GET',
path: '/delete/{id}',
handler: function (request, reply) {
var id = request.params.id;

return request.abase.db.delete(
{ tableName: 'posts', where: { id: id } },
function () { return reply.redirect('/') }
);
}
}, {
method: 'POST',
path: '/update/{id}',
handler: function (request, reply) {
var id = request.params.id;

return request.abase.db.update(
{ tableName: 'posts', where: { id: id }, fields: request.payload },
function () { return reply.redirect('/') }
);
}
}];
10 changes: 10 additions & 0 deletions example/schema.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
'use strict';

module.exports = {
tableName: 'posts',
fields: {
title: { type: 'string' },
body: { type: 'string' },
id: { type: 'number', integer: true }
}
};
19 changes: 19 additions & 0 deletions example_schema.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
'use strict';

module.exports = {
tableName: 'user_data',
fields: {
email: {
type: 'string',
email: true
},
username: {
type: 'string',
min: 3,
max: 20,
unique: true
},
dob: { type: 'date' },
id: { type: 'id' }
}
};
Loading