Use AWS API Gateway for External Access to an Existing Web App

My team’s software project has an existing Next web app that supports generating read-only reporting data about user activity in the system as a CSV download. Customers were requesting programmatic access to this data, so they could write a script to import that reporting data into their own CRMs.

API Gateway: Understanding our Use Case

We only wanted to expose a small chunk of the web app’s functionality and had no intention of expanding the API’s feature set going forward. And, we had about three weeks left in an already-short engagement when we picked up this chunk of work.

With that in mind, we wanted a solution that satisfied the following:

• Could be built and tested fairly quickly, and would not require a big refactor to accomplish.
• Optimize a better developer experience around the development of the core web app.
• Allow some other way to authorize access to reporting data. With the CSV export, an organization admin user would need to log in and generate the report. So, cookie auth was used to pick the user info off the session. With API access, we still need to scope requests to a given organization, but there is no user so this cookie auth approach does not apply.

Having considered those requirements, we decided to just expose a few Next API Routes rather than implementing, say, a cleaner solution that had shared components but was ultimately separate projects for our Next app vs. this new external API. We then created an API Gateway service to point towards these Next API routes, as a service for generating and managing API keys.

Setting Up the Resources

Next, I’ll walk through the different resources we set up. We created the following API Gateway resources directly in the AWS console itself:

• API Gateway REST API – We create a REST API, which allows us to configure any HTTP request handler (so, our Next app, in our case) as the backend/integration request for the endpoints exposed through the API Gateway API. REST API also has support for things like API keys, rate limiting, etc. Within this REST API, we can then create 2 stages – one for our development environment, and the other for our production environment. Each stage can have different stage variables. We use stage variables to route the stages to the correct web app backends with an integrationUrl stage variable.
• API Gateway Usage Plans and API Keys – Next, we set up two usage plans. Again, one for dev and one for prod. We also configured each usage plan with appropriate quotas and rate limiting. And finally, we can create API keys and assign them to the appropriate usage plans.

Configuring Endpoints with OpenAPI

Now, with an understanding of the different resources, I’ll walk through how we configured the endpoints that were exposed through our API Gateway API.

We used an OpenAPI specification document with AWS API Gateway extensions, which has a few benefits:

• This OpenAPI document can double as documentation that we can send to the developers consuming our API.
• If we need to add a new operation to the API, we do not have to rely on just remembering what buttons in the AWS console we clicked to arrive at the desired configuration.

Here is an example of how to extend an operation’s definition with API gateway configurations.

paths:
  /users:
    get:
      operationId: getUsers
      parameters:
        - $ref: "#/components/parameters/pageLimitParam"
        - $ref: "#/components/parameters/pageOffsetParam"
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/usersResponse"
      security:
        - ApiKeyAuth: []
      x-amazon-apigateway-integration:
        httpMethod: "GET"
        uri: "https://${stageVariables.integrationUrl}/api/users"
        responses:
          "200":
            statusCode: "200"
        requestParameters:
          integration.request.header.api-key-id: "context.identity.apiKeyId"
          integration.request.querystring.pageOffset: "method.request.querystring.pageOffset"
          integration.request.querystring.pageLimit: "method.request.querystring.pageLimit"
        passthroughBehavior: "when_no_templates"
        type: "http"
        parameters:
          - $ref: "#/components/parameters/pageLimitParam"
          - $ref: "#/components/parameters/pageOffsetParam"
components:
  parameters:
    ...
  schemas:
    ...
  securitySchemes:
    ApiKeyAuth:
      type: apiKey
      name: x-api-key
      in: header
      x-amazon-apigateway-api-key-source: HEADER

Here are a few things to note about this example configuration.

Using API keys to authorize API access

Each generated API Key consists of an ID in addition to the actual secret value for the key. We send the actual key to developers integrating with our API. We then configure API Gateway to transform the API key it receives into an API Key ID so that our request handler can identify who owns the key used to determine the correct data to return. This translation happens in the x-amazon-apigateway-integration.requestParameters.

The x-api-key header must be defined as a security scheme. This is how the gateway knows the key is required on the method request. If the API key is not tagged as required, the context.identity.apiKeyId will always be null, regardless of whether the incoming requests include it or not.

Fully Defining the Integration

If you define the x-amazon-apigateway-integration at all, you must fully define the integration properties.

If you do not include all request params on this integration object, then any request params not documented in the integration sent to the method request (what gateway receives) will be lost when passing off to the integration request (what our Next app receives).

The same applies to responses. If you only, say, include the 200 response on the x-amazon-apigateway-integration.responses, API Gateway will transform a 400 response from your server into a 500 error, since it didn’t know about the possibility of your server sending a 400.

This is true even if you do specify these request params/responses as part of the base OpenAPI spec — that is only for defining the method request.

Ideas for Improvements

This approach came together quickly, but it’s definitely not a perfect solution. Here’s a to-do list of some possible enhancements we could make:

• Having to fully define the API Gateway integration is a bit tedious. What about APIs that are supposed to be direct pass-throughs except for just the API key transformation, which is common to all operations? What support is there for global transformation policies?
• It would be great to further limit access to these Next API routes, to enforce that any requests to those routes must come from the API gateway. With a standalone web server, we could pull in something like incoming IP restrictions or client certificates. How might we be able to similarly restrict access for certain API routes in our Next project?
• I mentioned earlier that we use the OpenAPI spec file for both API Gateway configuration and documentation. With that, we must strip out any gateway config details. The AWS console has support for exporting an OpenAPI spec with API Gateway extensions removed. But, with this, we would also lose a lot of the descriptions we wrote. Instead, could we set up a little script to delete content for any paths related to Amazon API Gateway?