Skip to content

Commit

Permalink
Stable release: 1.0.0 (#108)
Browse files Browse the repository at this point in the history
* [#75] patching things to get clearer/easier deploy-to-production steps

* A second fix on production to make b2handle credentials easier

* Bug fix

* Testing one last time

* [#77] First attempt to fix all imports

* Bug fix on travis virtualenv

* bug fix on prerequisites script

* fix permissions for coverage on travis

* [#76] fixing coveralls jobs with a manual entry of the token

* [#76] test bug fix for linking the relative paths

* [#76] bug fix on working dir inside docker for autobuild with coverage

* author requested removal ^_^

* [#76] Testing working coverage from rapydo command

* Adding coveralls to travis...

* typo

* Testing a coverage fix

* [#76] Adding more badges

* Creating the new branch for release candidate

* Skip init for coverage

* Adding coverage empty file to collection

* Testing coveralls again on travis

* update to latest ubuntu supported by travis

* coverage travis bug fix

* [#] Drafting an endpoint to test the b2safe irods authentication

* Moving the endpoint to auth and adjusting unittest

* [#83] making the login save the user as cache and produce a token

* [#83] debugging b2safe proxy

* Updating part of the quick start

* [#83] Towards rc1 and b2safe proxy

* Working on the docs

* Add chat from gitter badge, for RC1

* Completing serious unit tests on b2safeproxy

* Updating to rapydo 0.5.3

* Updating quick start documentation

* Writing more documentation

* Testing and updating the docs

* Updating changelog

* cleaning issue styles on changelog

* Updating to the latest irods build

* Preparing the rc2 configuration and the new irods

* Updating changelog

* [#88] Reorganizing code to select authentication based on token/db

* Bug fix on path with no permissions

* Fixed upload with pid=true and no file name passed in URL

* removed prints

* [#94] Fixing the certificate path

* Uniform reply from b2safe proxy as we already do with b2access auth

* Adding the upgrade tip on current quick start

* more tips

* bug fix

* Bug fix: supposely you access the b2safe system with normal credentials

* [#95] First draft of a client template

* [#97] Considering basic auth digest with the b2safe auth proxy

* [#95] Listing directory with files to be pushed to HTTP API

* [#95] Working upload of file with the template client

* [#95] Moving helper functions inside rapydo utils

* documentation updated

* documentation updated

* [#96] Testing irods failing with multiple zones.

* Updating instructions

* Revisiting the concept of admin/guest for debug/production

* drafting irods standalone with docker

* Preparing the rc2 release

* [#95] Fixing problems with client upload/patch

* Delete directory and create it again

* Bug fix

* Delete directory and create it again

* [#92] First draft. Proxy yet to be checked.

* Resolve PIDS via pids endpoint

* [rapydo/issues#61] Custom 502 page configuration

* Cosmetic changes

* Updating with latest helper

* Testing production mode more

* Updates to changelog

* Ignoring backups on upgrade

* Cosmetic changes

* Getting ready for official RC2

* The new milestone

* cosmetic change

* Update to latest rapydo version 0.5.5

* Script to uplod (PUT) in parallel

* Removed request.get_datat() which empties upload data stream

* Switch to rapydo 0.5.6

* Preparing switch from 'eudat' project to a more specific 'b2stage'

* Upgrading path to latest rapydo 0.5.6 version

* [#20] Adding a pool size parameter to change in production.

* Adding copyrights

* Setting production timeouts for nginx

* Update license as EUDAT compliant service

* Renaming "eudat" with a more appropriate "b2stage".
Creating a "b2safe" sub-project for docker images.

* Adding b2safe official image inside the project

* Adding a readme for b2safe dockerized

* Retreive other optional PID fields

* skip files in data

* Removing buffer from nginx proxy

* [#20] Update postgres and adminer

* [#106] Refactoring pids calls and unittests.

* [#105] Completing fixes for new unittests of rapydo

* [#20] Adding a new configuration for postgresql in production.

* [#20] Fixing the postgresql bug

* Adding b2safe layer

* Update instructions

* typo...

* [#72] enabling irods anonymous user to the default docker image

* [#72] allow login to anonymous with no password. With (new) unittests.

* [#72] A draft for making our data public

* bug fixes

* Testing b2access integration/staging server

* [#72] refactor to recursive apply permissions to parent directories

* [#72] Better paths for publishing

* [#72] Drafting first part of the unittest (get method)

* Fixing parenthesis in old tests

* [#72] Completed the set of unittests.

* Completing publish endpoint

* [#79] Enabling the right services when really possible for b2access

* [#79] Completed the dependencies for b2access/b2safe proxies

* Bug fix

* [#104] Fix bug in production to enable certificates

* [#103] Better configuration for production restart on failure

* [#95] Complete the latest operations on the client template

* Change pid parameter to 'pid_await'

* [#42] Draft of a public endpoint as a landing page for browsers

* Switching to 1.0.0 and creating a changelog

solves #107

* [#99] Updating the documentation with a new index

* Adding the (empty) folder for b2host seadata capabilities

* Improved PUT doc

* Fixed links

* Update to latest rapydo. Tested b2access dev with new certificates.

* bug fix
  • Loading branch information
pdonorio authored Oct 27, 2017
1 parent 481ea00 commit fa8db95
Show file tree
Hide file tree
Showing 122 changed files with 1,994 additions and 927 deletions.
16 changes: 8 additions & 8 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,31 +6,31 @@
# data
uploads
tmp_files
b2stage/*
# virtualenv

# Skip self-signed certificates
b2stage/*
data/certs/nginx-selfsigned*
data/certs/*cineca*
data/secrets/secret_*
data/b2handle/*.pem
data/files/*

#################
## Submodules ##
#################

submodules/.backup
submodules/utilities
submodules/builds_base
submodules/backend
submodules/build-templates
submodules/utils
submodules/http-api
submodules/frontend
submodules/prc

############################################
## Dynamically created file and folders ##
############################################

.projectrc
frontend/eudat/libs/bower_components
#.projectrc
frontend/b2stage/libs/bower_components
.env
projects/*/backend/confs/project_configuration.yaml

Expand Down
6 changes: 6 additions & 0 deletions .projectrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#log-level: VERY_VERBOSE
#log-level: VERBOSE
mode: debug
project: b2stage
#mode: production
#hostname: b2stage.cineca.it
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ python:
services:
- docker
env:
- PROJECT=eudat
- PROJECT=b2stage

# # installation
# before_install:
Expand Down
29 changes: 18 additions & 11 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,30 @@

# v0.7.0-rc4 (*near future*)
**due date**: October 15th, 2017
# v1.0.1 (*near future*)
**due date**: November 30th, 2017

## features

- Connect prototype to production services #90
- Checklist for good projects #62
- to be defined
- to be defined in the next developer meeting

# v0.6.2-rc3 (*currently developed*)
**due date**: October 1th, 2017
**project**: [webinar](https://github.com/EUDAT-B2STAGE/http-api/projects/3)
# v1.0.0 (*first stable release*)
**due date**: October 24th, 2017
**milestone**: [stable release](https://github.com/EUDAT-B2STAGE/http-api/milestones/8)

## features

- Define a protocol to release production to be tested automatically #55
- to be defined
- Benchmarks #20
- Unittests upgrade (from `nose2` to `py.test`) #105
- A publish endpoint to make data available to anyone/anonymous #72
- PIDS refactor #106
- Automatically recover from failures #103
- Landing page #42
- Bug fixes

## known issues

Currently B2ACCESS and B2SAFE are facing problems in sharing certificated credentials. This is under investigation and will be discussed in the upcoming developer meeting. It is not an issue related to B2STAGE HTTP-API code.

# v0.6.1-rc2 (*latest stable*)
# v0.6.1-rc2
**due date**: September 9th, 2017
**project**: [release candidate 2](https://github.com/EUDAT-B2STAGE/http-api/milestone/9)

Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The MIT License (MIT)

Copyright (c) 2016 B2STAGE service core code for EUDAT project
Copyright 2011-2017 EUDAT CDI - www.eudat.eu

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
53 changes: 31 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,9 @@
# B2STAGE HTTP-API for EUDAT project


| build | coverage | quality | uptime | swagger |
| --- | --- | --- | --- | --- |
| [![Build Status](https://travis-ci.org/EUDAT-B2STAGE/http-api.svg?branch=master)](https://travis-ci.org/EUDAT-B2STAGE/http-api) | [![Coverage Status](https://coveralls.io/repos/github/EUDAT-B2STAGE/http-api/badge.svg?branch=HEAD)](https://coveralls.io/github/EUDAT-B2STAGE/http-api?branch=master) | [![Code Health](https://landscape.io/github/EUDAT-B2STAGE/http-api/master/landscape.svg?style=flat)](https://landscape.io/github/EUDAT-B2STAGE/http-api/master) | [![Uptime Robot](https://img.shields.io/uptimerobot/ratio/m778586640-4e31f2b00e90bce508dcdf33.svg?maxAge=2592000)](https://stats.uptimerobot.com/xGG9gTK3q) | [![Swagger validation](https://img.shields.io/swagger/valid/2.0/https/b2stage.cineca.it/api/specs.svg)](http://petstore.swagger.io/?url=https://b2stage.cineca.it/api/specs&docExpansion=none) |

NOTE: the B2STAGE HTTP-API is an interface towards other EUDAT services which are subject to modifications and new developments, therefore the functionalities and the implementation will have to be changed accordingly.

Feedback on our first Release Candidate: [![Gitter](https://badges.gitter.im/EUDAT-B2STAGE/http-api.svg)](https://gitter.im/EUDAT-B2STAGE/http-api?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
| build | coverage | quality | uptime | swagger | feedback |
| --- | --- | --- | --- | --- | --- |
| [![Build Status](https://travis-ci.org/EUDAT-B2STAGE/http-api.svg?branch=master)](https://travis-ci.org/EUDAT-B2STAGE/http-api) | [![Coverage Status](https://coveralls.io/repos/github/EUDAT-B2STAGE/http-api/badge.svg?branch=master)](https://coveralls.io/github/EUDAT-B2STAGE/http-api?branch=master) | [![Code Health](https://landscape.io/github/EUDAT-B2STAGE/http-api/master/landscape.svg?style=flat)](https://landscape.io/github/EUDAT-B2STAGE/http-api/master) | [![Uptime Robot](https://img.shields.io/uptimerobot/ratio/m778586640-4e31f2b00e90bce508dcdf33.svg?maxAge=2592000)](https://stats.uptimerobot.com/xGG9gTK3q) | [![Swagger validation](https://img.shields.io/swagger/valid/2.0/https/b2stage.cineca.it/api/specs.svg)](http://petstore.swagger.io/?url=https://b2stage.cineca.it/api/specs&docExpansion=none) | [![Gitter](https://badges.gitter.im/EUDAT-B2STAGE/http-api.svg)](https://gitter.im/EUDAT-B2STAGE/http-api?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) |


---
Expand All @@ -23,22 +19,37 @@ The primary goal is to allow users to ingest and retrieve data via a standard RE
- lower the entry barrier to using EUDAT services,
- simplify integration into existing workflows,
- allow direct access to data assets held with the EUDAT CDI.

The first development is focused on the interaction with B2SAFE, allowing users to transfer and manage data on the "registered" domain.
<!--
Over the EUDAT2020 project other functionalities will be added: the development road map is available on the [EUDAT Wiki](https://confluence.csc.fi/display/EUDAT2/Service+building+roadmap)
-->


## Documentation

Documentation for the latest stable release:

- [Quick start](docs/user/user.md) for deploy and development
- [Deploy](docs/deploy/deploy.md) the server
- [User](docs/user/user.md) guide
- [Developer](docs/development/development.md) instructions
The first stable release is focused on the interaction with the `B2SAFE` service, allowing users to transfer and manage data on the `registered` **domain**.

- Get started
+ try a [quick start](docs/user/user.md)
+ see the latest [webinar](https://pdonorio.github.io/chapters/webinars/b2stage)
+ a demo [prototype](docs/prototype.md) of the current functionalities
- User guide
+ [use](docs/user/user.md) an existing instance of the `HTTP-API`
+ [main authentication](docs/user/authentication.md) based on `B2ACCESS`
+ [alternative authentication](docs/user/authentication_b2safe.md) based on `B2SAFE`
+ [endpoints](docs/user/endpoints.md) description and examples
- Administration
+ read the [pre-requisites](docs/deploy/preq.md) prior to installation
+ [startup](docs/deploy/startup.md) a working copy
+ deploy in two [modes](docs/deploy/modes.md)
+ [debug](docs/deploy/debugging.md) the server operating on top of `B2SAFE` and fix eventual issues
- Development
+ [add](docs/development/development.md) new features
+ extra [operations](docs/development/operations.md)

<!--
Documentation for the current developed branch instead can be found
[here](https://github.com/EUDAT-B2STAGE/http-api/tree/0.6.2/docs).
-->


## Implementation
Expand All @@ -48,13 +59,11 @@ This project is based on the open-source [RAPyDo framework](https://github.com/r
The API server is implemented with the latest Python 3 release using the Flask core. To facilitate and speed the development Docker is the base environment to automate the creation and configuration of the software stack needed.
The API interconnects with EUDAT services' native APIs or libraries.

To exchange data and metadata within the B2SAFE service (built on iRODS system as backend storage) the interface is implemented using the official [python-irodsclient](https://github.com/irods/python-irodsclient).
To exchange data and metadata within the B2SAFE service (built on iRODS system as backend storage) the interface is implemented using (and contributing to) the official [python-irodsclient](https://github.com/irods/python-irodsclient).

<!--
## quick notes
NOTE: *the B2STAGE HTTP-API is an interface towards other EUDAT services which are subject to modifications and new developments, therefore the functionalities and the implementation will have to be changed accordingly*.

b2access basic endpoints:
https://eudat.eu/services/userdoc/b2access-service-integration#B2ACCESS_Services_Endpoints

https://eudat-aai.fz-juelich.de:8445/oauth-demo/get_token.jsp
-->
## Copyright

Copyright 2011-2017 EUDAT CDI - www.eudat.eu
Loading

0 comments on commit fa8db95

Please sign in to comment.