Proposal: Implementable Documentation

[mbleigh]

I've been pondering on this one for a while, so I'm just going to lay out some of the fundamentals I've been thinking about and go from there. Currently we have desc and params and some arbitrary conventions that help feed grape-swagger. I think we can do better. This proposal is somewhat based on some hacks we've done at Divshot and partially based on what I think makes sense.

The heart of this proposal is making desc a block-accepting method that yields a documentation DSL. With this DSL, params may be (optionally) nested inside the desc block or provided separately.

Standardized Error Format

It is useful to have both human and machine readable error conditions for APIs. Currently Grape has a simple {"error":"<message>"} format but this is often not enough information by itself. I propose the following format:

{
  "error":"machine_readable_code",
  "error_description":"Human readable error description goes here.",
  "context": {
    "optional":"Information about the error goes here."
  }
}

This format is compatible with the OAuth 2.0 error response formats (which is why I named things as I did) with the addition of context, which might provide (for example) which parameters were invalid.

This format would replace the current default error formatter and in an endpoint would look something like:

error! 400, :something_bad, "Something bad has happened, get help!"

I'm moving the status to the front of the queue here both because it allows us to potentially implement in a backwards-compatible way and because the HTTP Status is one of the most important pieces of info that can be communicated and I now think it was a mistake to make it an optional parameter. You should always think about and provide a sane HTTP status on error.

Declarative Response Modes

For the purposes of documentation, automated tooling, etc. it is very useful to know what the characteristics of all possible responses to an endpoint are. I propose that the desc DSL include success and failure methods that work like so:

desc "Post a status update." do
  detail "This would be a much longer description of what the endpoint does, perhaps spanning multiple lines etc."
  success Tweet::Entity, 201, "The newly created status update."
  failure :not_logged_in, 401, "You must be logged in to perform that action."

  params do
    requires :text, 
  end
end
post :status do
  fail! :not_logged_in unless current_user
  present Tweet.create(params.text)
end

I believe that a DSL like this will create great at-a-glance documentation in the code itself while also providing all the metadata needed to generate documentation. Unlike the existing "do whatever you want in the desc hash" setup, the failure modes and success modes can actually affect endpoint behavior.

The fail! method works much like error! except it only allows failure modes that have already been defined in desc blocks.

Reusing Failure Modes

It will often be convenient to reuse a failure mode. This will be done the same way that it is today with params:

desc do
  failure :admin_required, 401, "You must be an administrator to access this resource."
end
namespace :admin do
  before{ fail! :admin_required unless current_user.admin? }
end

Of course you can do this on scope as well to avoid creating a new namespace:

desc do
  failure :authentication_required, 401, "You must be logged in to access that resource."
end
scope :authenticated do
  before{ fail! :authentication_required unless current_user }
end

Entity Definition

The second argument of the success call above is an entity class. The only thing required of an entity class is that it respond to entity_schema and return a hash that looks something like this:

{
  name: 'User',
  desc: 'A user of the system.',
  attributes: {
    attr_name: {type: String, desc: 'A brief description of the thing', example: 'Sample Value', default: 'Default Value'}
  }
}

This would be built in to Grape::Entity obviously, but could be easily implemented in other representation systems as well.

Markdown and More

The DSL above provides great facilities for terse description of API endpoints. However, for more in-depth content putting that in the .rb files doesn't really make sense. This is less thought through, but here's one option:

1. Anywhere that a string can be passed as descriptive text in the desc DSL, also accept a symbol.
2. Have a means of loading descriptive text based on symbols (this also opens up i8n which is nice)

This should basically work by loading a big hash, as you'd expect.

What I think I would want to do, personally, is build a system based on Markdown that would parse out H1s to get key names. I was going to go into detail but I think that actually makes more sense as a separate library.

Expanding Documentation Output

With a more robust documentation system built into Grape, it would be easy to support all of the various API description libraries (we have Swagger now, but could then support Blueprint and others without much trouble).

OK, that's the longest issue description I've ever written. Thoughts? I want some feedback before I'd start diving into implementation.

[idyll]

We are currently using the grape/swagger combination with a bunch of clients - about a dozen projects - and I would welcome these changes.

Right now the documentation kinda obfuscates the code when it is present inline. And I've had to make customizations to swagger to make it aware of custom HTTP headers that we us for authentication instead of the basic or token auth they support.

A fresh start like this would help us clean up a lot of code.

[dblock]

I wanted to add a +1 to this.

[dmitry]

Hopefully it will be completed and merged.

[rymai]

Awesome proposal @mbleigh ! 👍

[swistaczek]

Hey, this idea is great. This will allow us to keep extensions like grape-swagger less hacky. 👍

[pragmaticivan]

Awesome proposal @mbleigh ! 👍

[toshe]

👍 Great proposal!
Any plans on merging?

[dblock]

The code in this PR is definitely nowhere near the detailed description here. So no plans on merging this "as is".

[mbleigh]

Yeah, this is something I just haven't had any time to work on to my continued frustration. If someone else is really interested in implementing, I would be super-excited! Otherwise, I'll try to find time somehow, someday.

[toshe]

@dblock ok!
I've expanded https://github.com/connexio-labs/grape-markdown and grape-slate to include an option to display sample Entity output based on the success paramater in the desc block. Basically what it does takes the exposures from the Entity Class and generates sample output based on the exposure names (including :as) and the examples specified in the documentation block. But this works only if a grape-entity is specified.
@mbleigh I really like your idea about expanding the entities (and other presenters/views for that matter) with an entity_schema. Our API is using several "types" of the same Entities, so perhaps it might be a good idea to also specify the type of the Entity inside the success/failure param in the desc DSL. This should also be reflected in the schema_info hash. Also this schema_info should include the final name exposed (especially when specifying :as). Thoughts on this?
Also what about cyclic/nested exposures? Should the schema_info be expanded or provide symbols?

Perhaps it would be more practical for the entity class to respond to a sample_output function where you could optionally specify :root and :type ...

[dblock]

What should we do about this PR? Bump @mbleigh.

[dblock]

I am going to close this. This can definitely be implemented today as a Grape extension if someone wants to take it further into a gem.