Skip to content

Commit

Permalink
Merge pull request #15 from DavidePrincipi/subscription
Browse files Browse the repository at this point in the history
 Refactor for Porthos multi-tenant configuration

nethesis/dev#5641
  • Loading branch information
DavidePrincipi authored Jun 27, 2019
2 parents aa1c833 + bd952d4 commit 56fa946
Show file tree
Hide file tree
Showing 14 changed files with 436 additions and 100 deletions.
63 changes: 54 additions & 9 deletions porthos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,34 +68,79 @@ repository mirrors for the given parameters.
`true`
- `arch` system architecture, like `x86_64`

HTTP repository metadata query (HTTP Basic authentication required):
HTTP repository metadata query (SSL + HTTP Basic authentication required):

https://m1.nethserver.com/autoupdate/<repo_version>/<repo_name>/<repo_arch>/repodata/repomd.xml
https://m1.nethserver.com/stable/<repo_version>/<repo_name>/<repo_arch>/repodata/repomd.xml

HTTP repository metadata query (username as path token, SSL not required):

http://m1.nethserver.com/autoupdate/<username>/<repo_version>/<repo_name>/<repo_arch>/repodata/repomd.xml
http://m1.nethserver.com/stable/<username>/<repo_version>/<repo_name>/<repo_arch>/repodata/repomd.xml


* `autoupdate` returns data from the tier associated to the credentials provided
* `stable` always returns data from `t0`

## HTTP status codes

* 401 - authorization required
* 404 - resource not found
* 403 - bad credentials or `tier_id` is false (disabled)
* 403 - bad credentials or access denied conditions
* 503 - redis connection failed, see `/var/log/nginx/porthos-php-error.log`
* 502 - php-fpm connection failed, see `/var/log/nginx/error.log`
* 500 - generic PHP error, see `/var/log/nginx/porthos-php-error.log`

## Redis DB format
## Porthos repository access control

Access permissions to Porthos repositories are checked by `auth.php`.

A special `subscription.json` API endpoint used by the Software center clients
is implemented by `subscription.php`.

The `repomd.php` script expects the following storage format in redis DB:
All HTTP requests to access YUM repositories must be authenticated with HTTP
Basic Auth. If the special `$config['legacy_auth']` is enabled, the HTTP
username is considered a valid access token, and can be set as password value too.

The access token can be passed also as an URL path component, like:

http://m1.nethserver.com/stable/<username>/<repo_version>/<repo_name>/<repo_arch>/repodata/repomd.xml

The `auth.php` and `subscription.php` scripts expects the following record
format in redis DB:

key: <system_id>
value: hash{ tier_id => <integer>, secret => <string> }
value: hash{ tier_id => <value>, secret => <value>, icat => <value> }

If `tier_id` is not set, the access is denied (403 - forbidden). For instance to create a key on athos
For instance to create a key on athos

redis-cli -p PORT
redis-cli PORT> HMSET 0ILD29RH-D78A-C444-1F82-EE92-3211-FC47-43AD-DQFD tier_id 2 secret S3Cr3t
redis-cli PORT> HMSET 0ILD29RH-D78A-C444-1F82-EE92-3211-FC47-43AD-DQFD tier_id 2 secret S3Cr3t icat cat1,cat2,cat3

### `tier_id` field

The `tier_id` value should be a number. If the value is negative, the tier
number is calculated by an hash function, based on the system identifier.

If `tier_id` is not a number, both `auth.php` and `subscription.php` reply with
403 - forbidden.

### `icat` field

The `icat` field is a string of a comma separated list of YUM category
identifiers (refer to the repository comps/groups for valid names). Its purpose
is to show the products entitlement on the Software center page. It is used by
`subscription.php` to return the included/excluded YUM categories list to the
client. See also the `$config['categories']` parameter to configure it. This
field is ignored by `auth.php`.

If `icat` field is not set, the `subscription.php` replies with 403 - forbidden.

### `secret` field

If `secret` field is not set, both `auth.php` and `subscription.php` reply with
403 - forbidden, unless `$config['legacy_auth']` is enabled.


## Repository management commands

Expand All @@ -104,10 +149,10 @@ from `/etc/porthos.conf`. Upstream YUM rsync URLs are defined there.

- `repo-bulk-hinit` runs initial synchronization from upstream repositories (-f disables the check for already existing directories)
- `repo-bulk-pull` creates a snapshot date-dir (e.g. `d20180301`) under
`dest_dir` with differences from upstream repositories. Set `t0` to point at
`dest_dir` with differences from upstream repositories. It sets `t0` to point at
it.
- `repo-bulk-shift [N]` updates `t1` ... `tN` links by shifting tiers of one position
the optional `N`
the optional `N` parameter creates missing links up to N - 1.
- `repo-bulk-cleanup` erases stale tier snapshots

The following commands should not be invoked directly. They are intended to be
Expand Down
25 changes: 0 additions & 25 deletions porthos/root/etc/nginx/conf.d/mirrorlist.conf

This file was deleted.

53 changes: 41 additions & 12 deletions porthos/root/etc/nginx/conf.d/porthos.conf
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ server {
listen 80;
listen [::]:80;

server_name ~^m[0-9]+.nethserver.com;
server_name porthos.nethserver.com;

# Uncomment the following line when TLS certificate is available
# include porthos-certbot.conf;
Expand Down Expand Up @@ -38,36 +38,65 @@ server {
return 200 "pong\n";
}

location ~ "^/t0/(.*)" {
location ~ "^/T0/(.*)" {
internal;
try_files $uri /head/$1;
try_files /t0/$1 /head/$1;
}

location ~ "^/t1/(.*)" {
location ~ "^/T1/(.*)" {
internal;
try_files $uri /t0/$1;
try_files /t1/$1 /t0/$1 /head/$1;
}

location ~ "^/t2/(.*)" {
location ~ "^/T2/(.*)" {
internal;
try_files $uri /t1/$1;
try_files /t2/$1 /t1/$1 /t0/$1 /head/$1;
}

location ~ "^/t3/(.*)" {
location ~ "^/T3/(.*)" {
internal;
try_files $uri /t2/$1;
try_files /t3/$1 /t2/$1 /t1/$1 /t0/$1 /head/$1;
}

location ~ "^/(d[0-9]{8}|head)(/.*)" {
# add another location for tier "Tx" here...

location ~ "^/(t[0-9]|d[0-9]{8}|head)(/.*)" {
internal;
}

location ~ "^/(autoupdate|stable)/(.*)" {
include fastcgi.conf;
fastcgi_pass unix:/var/run/porthos-fpm;
fastcgi_param SCRIPT_FILENAME /srv/porthos/script/auth.php;
fastcgi_param PORTHOS_SITE porthos;
fastcgi_param PORTHOS_REDIS /var/run/redis/athos.sock;
fastcgi_param DOCUMENT_URI /$2;
fastcgi_param DOCUMENT_URI $uri;
}

}

server {
listen 80;
listen [::]:80;

server_name porthos-mirrorlist.nethserver.com;

# Uncomment the following line when TLS certificate is available
# include porthos-certbot.conf;

access_log /var/log/nginx/mirrorlist.access.log main;
root /usr/share/nginx/html;

location /.well-known/acme-challenge {
root /srv/porthos/certbot;
allow all;
}

}
location / {
include fastcgi.conf;
fastcgi_pass unix:/var/run/porthos-fpm;
fastcgi_param SCRIPT_FILENAME /srv/porthos/script/mirrorlist.php;
fastcgi_param PORTHOS_SITE porthos;
fastcgi_param PORTHOS_REDIS /var/run/redis/athos.sock;
}
}
1 change: 0 additions & 1 deletion porthos/root/etc/redis-athos.conf
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
# do not listen on TCP sockets
port 0

unixsocket /var/run/redis/athos.sock
unixsocketperm 660

# connect athos from local stunnel endpoint
Expand Down
10 changes: 6 additions & 4 deletions porthos/root/etc/systemd/system/[email protected]
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,14 @@ ConditionPathExists=/etc/redis-%i.conf

[Service]
Type=notify
ExecStart=/usr/bin/redis-server /etc/redis-%i.conf --daemonize no
ExecStop=/usr/bin/redis-cli -s /var/run/redis/%i.sock shutdown
Environment="UNIXSOCK=%t/redis-%i/server.sock"
ExecStartPre=/usr/bin/chcon -L -u system_u -t redis_var_run_t %t/redis-%i
ExecStart=/usr/bin/redis-server /etc/redis-%i.conf --unixsocket $UNIXSOCK --loglevel debug --daemonize no
ExecStop=/usr/bin/redis-cli -s $UNIXSOCK shutdown
User=redis
Group=nginx
RuntimeDirectory=redis
RuntimeDirectoryMode=0755
RuntimeDirectory=redis-%i
RuntimeDirectoryMode=0750
LimitNOFILE=10240

[Install]
Expand Down
102 changes: 89 additions & 13 deletions porthos/root/srv/porthos/script/auth.php
Original file line number Diff line number Diff line change
@@ -1,27 +1,103 @@
<?php

include("lib.php");
/*
* Copyright (C) 2019 Nethesis S.r.l.
* http://www.nethesis.it - [email protected]
*
* This script is part of Dartagnan.
*
* Dartagnan is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License,
* or any later version.
*
* Dartagnan is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with Dartagnan. If not, see COPYING.
*/

if ( ! isset($_SERVER['PHP_AUTH_USER'])) {
header('WWW-Authenticate: Basic realm="subscription"');
header('HTTP/1.1 401 Unauthorized');
echo "Provide system subscription credentials\n";
exit;
require_once("lib.php");
require_once("config-" . $_SERVER['PORTHOS_SITE'] . ".php");

$uri = parse_uri($_SERVER['DOCUMENT_URI']);

if ( isset($_SERVER['HTTPS']) && ! $uri['system_id'] && ! isset($_SERVER['PHP_AUTH_USER'])) {
exit_basic_auth_required();
} else {
if($uri['system_id']) {
// override PHP authentication with system_id token:
$_SERVER['PHP_AUTH_USER'] = $uri['system_id'];
$_SERVER['PHP_AUTH_PW'] = $uri['system_id'];
}
}

// Disable the Content-Type header in PHP, so that nginx x-accel can add its own
ini_set('default_mimetype', FALSE);

$access = get_access_descriptor($_SERVER['PHP_AUTH_USER'], $_SERVER['PHP_AUTH_PW']);
// Mask any repo/version/arch that does not belong to the site:
if(! in_array($uri['repo'], $config['repositories'])
|| ! in_array($uri['version'], $config['versions'])
|| ! in_array($uri['arch'], $config['arches'])) {
exit_http(404);
}

if($access['tier_id'] === FALSE) {
if(! isset($_SERVER['PHP_AUTH_USER']) || ! isset($_SERVER['PHP_AUTH_PW'])) {
exit_http(403);
} elseif ($access['tier_id'] === '' || $access['secret'] != $_SERVER['PHP_AUTH_PW']) {
}

$access = get_access_descriptor($_SERVER['PHP_AUTH_USER']);
$valid_credentials = $_SERVER['PHP_AUTH_PW'] === $access['secret'];
if($config['legacy_auth']) {
$valid_credentials = $valid_credentials || $_SERVER['PHP_AUTH_USER'] === $_SERVER['PHP_AUTH_PW'];
}
$has_access_disabled = ! is_numeric($access['tier_id']);
if ($has_access_disabled || ! $valid_credentials) {
exit_http(403);
} else {
if(basename($_SERVER['DOCUMENT_URI']) == 'repomd.xml') {
header('Cache-Control: private');
}

if($access['tier_id'] < 0) {
$hash = 0;
foreach(str_split($_SERVER['PHP_AUTH_USER']) as $c) {
$hash += ord($c);
}
return_file('/t' . $access['tier_id'] . $_SERVER['DOCUMENT_URI']);
$hash = $hash % 256;
if($hash < 12) { // 5%
$tier_id = 0;
} elseif($hash < 38) { // 15%
$tier_id = 1;
} elseif($hash < 76) { // 30%
$tier_id = 2;
} else { // 50%
$tier_id = 3;
}
$tier_id += $config['tier_id_base'];
} else {
$tier_id = intval($access['tier_id']);
}

if(basename($uri['rest']) == 'repomd.xml') {
header('Cache-Control: private');
application_log(json_encode(array(
'application' => 'porthos-' . $_SERVER['PORTHOS_SITE'],
'connection' => $_SERVER['CONNECTION'] ?: '',
'msg_type' => 'repomdxml-auth',
'msg_severity' => 'notice',
'server_id' => $_SERVER['PHP_AUTH_USER'],
'repo' => $uri['repo'],
'version' => $uri['version'],
'arch' => $uri['arch'],
'tier_id' => $uri['prefix'] == 'autoupdate' ? NULL : $tier_id,
'tier_auto' => isset($hash),
'tls' => isset($_SERVER['HTTPS']),
)));
}

if($uri['prefix'] == 'autoupdate') {
return_file('/T' . $tier_id . $uri['full_path']);
} else {
return_file('/head' . $uri['full_path']);
}
Loading

0 comments on commit 56fa946

Please sign in to comment.