Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Prototypes #37

Merged
merged 34 commits into from
Jul 8, 2020
Merged

RFC: Prototypes #37

merged 34 commits into from
Jul 8, 2020

Conversation

vito
Copy link
Member

@vito vito commented Sep 19, 2019

Rendered

fixes #7
supercedes #24

vito added 28 commits April 3, 2019 15:47
the proposal itself is not yet fleshed out at all - still in the
information collecting phase so that the motivations are clear

Signed-off-by: Alex Suraci <[email protected]>
...plus a bunch of tweaks to make the markdown linter happy

Signed-off-by: Alex Suraci <[email protected]>
I decided to stray from the initial v2 doc, which had a 'reset' event
type and a 'discovered' event type. Instead 'check' only returns a
stream of JSON objects containing config and metadata.

Signed-off-by: Alex Suraci <[email protected]>
Since `put` can both create and delete configs, it seems better to keep
them all using the same event structure, even if 'check' can only emit
one type of event (`discovered`).

The alternative to this would be to split deleting out of 'put'. At that
point they would each emit a single type of event, and the interface
would be less open-ended. There's value in that, but that value has to
be weighed against the value of being able to create + delete in one
action. It's also not super clear that all resource types would support
'delete'. (But then, not all of them will support 'put', either.)

Signed-off-by: Alex Suraci <[email protected]>
Signed-off-by: Alex Suraci <[email protected]>
this removes 'event types' from the response from each action, and
now they all have the same response.

event types were needed because 'put' could both create and delete, and
we needed a way to tell the difference. so instead, we've split 'delete'
out, and now each action has only one way to interpret the events, so
the types are no longer needed.

this makes the interface a bit less flexible, but it seems worth seeing
how far we can go with this constraint for the sake of having a simpler,
more focused interface. if we can go the distance with this limitation
it feels like stronger validation of the resources concept.

along the way I've put the configuration for each command under an
'actions' field in the info response. i'd like to also have it so
actions are optional, i.e. you can have a resource that doesn't
implement 'delete' or 'put'. this seems like it'll be necessary if we
want a common resource interface to be used for things like
notifications or triggers where there's nothing to really delete, and
even today there are cases where resources are 'read-only' and don't
support put at all (e.g. RSS).

Signed-off-by: Alex Suraci <[email protected]>
we were already relying on image semantics here by expecting it to be in
the working directory, so relying on CMD should be no worse.

this will allow greater flexibility for resource type authors, allowing
for a single-binary entrypoint rather than forcing them to have an
'info' executable in a particular location.

Signed-off-by: Alex Suraci <[email protected]>
Signed-off-by: Alex Suraci <[email protected]>
check no longer emits 'reset', it emits the given version first to show
it still exists (just as today)

Signed-off-by: Alex Suraci <[email protected]>
these all go hand-in-hand so they have the same RFC number and are in
the same PR, but they each deserve their own proposals

Signed-off-by: Alex Suraci <[email protected]>
this will allow us to have a common mechanism for configuring TLS across
all resource types. having it be a formal part of the interface feels
sensible as all resources are meant to interact with external state -
and in doing so, TLS will be a commonplace.

a later RFC (or perhaps another proposal within this one) will
demonstrate how users configure TLS - it may be something like the
pipeline-level, or another field alongside 'source' on the resource
definitions.

Signed-off-by: Alex Suraci <[email protected]>
this is not well-defined yet and this is easy enough to do as a later
iteration of the resource interface, now that it's versioned.

Signed-off-by: Alex Suraci <[email protected]>
Signed-off-by: Alex Suraci <[email protected]>
* introduce artifacts: and how it can be used as a migration path for
  explicit `get` after `put`
* update open/closed questions (they are all now closed)

Signed-off-by: Alex Suraci <[email protected]>
each of these proposals should be submitted as separate RFCs. each RFC
should show how the v2 *and* v1 interfaces are to be interpreted.

i've also removed the artifact resources proposal, which I initially
included in order to provide a concrete example relating today's
resource usage to the v2 interface. however, it can technically be done
independently of the v2 interface, if not just as a way of achieving
'explicit `get` after `put`' in a backwards-compatible way. so let's
just propose it separately and link to all four to demonstrate how it is
just one of four proposed interpretations.

Signed-off-by: Alex Suraci <[email protected]>
these are being made independent of v1 vs. v2

Signed-off-by: Alex Suraci <[email protected]>
calling this something that sounds like better english until someone
has a good reason otherwise

Signed-off-by: Alex Suraci <[email protected]>
@tremblaysimon
Copy link

I don't know if I am out of scope with my comment, but what will be the place of tasks with advent of prototype?

Does it will make sense to continue to do generic task using a pattern like this one exposed here: https://github.com/18F/cg-pipeline-tasks ?

@siennathesane
Copy link

@tremblaysimon I'm currently on the team that owns the repo you are referencing. Please feel free to ask any questions if you have them.

// cc @bengerman13 @hillaryj

@tremblaysimon
Copy link

@mxplusb, Thank you for sharing this! I referenced your repo mostly because it's the same pattern we are using at my company (with a private (unfortunately) "concourse-tasks" repo).

I was wondering if it will continue to make sense to use that type of generic task when prototype will be there.

In the end, regarding what I understand I think a prototype (like a resource-type today) will be easier to share and use for the consumers.

037-prototypes/proposal.md Show resolved Hide resolved
037-prototypes/proposal.md Show resolved Hide resolved
037-prototypes/proposal.md Show resolved Hide resolved
037-prototypes/proposal.md Show resolved Hide resolved
@vito
Copy link
Member Author

vito commented May 5, 2020

@tremblaysimon Sorry, didn't notice your question there! It is indeed a goal that Prototypes fill the need for generic re-usable tasks. The Re-inventing resource types blog post demonstrates how. 🙂

The long-term trend I want to see play out is away from tasks and towards prototypes. I think tasks will have their place as the first step: I just want to write a script and run it in an image. In a lot of situations you probably won't have a need to go past this step. Frankly a lot of CI scripting is ad-hoc.

As patterns emerge, prototypes for common tasks can start to take shape. For example, a ruby prototype may emerge which provides basic testing facilities, perhaps even gem building. A go prototype could be made for running tests, building binaries, and maybe even monitoring dependencies for new versions. The goal is to gradually replace "glue code" with testable, reusable, shareable prototype implementations.

Comment on lines +92 to +93
* Standardize TLS configuration so every resource doesn't implement their own
way: [concourse/rfcs#9](https://github.com/concourse/rfcs/issues/9)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some discussion of this in concourse/concourse#5190, for reference.

@vito vito merged commit 914f486 into concourse:master Jul 8, 2020
@vito vito deleted the prototypes branch July 8, 2020 18:10
@aoldershaw aoldershaw mentioned this pull request Jul 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants