Article summary
Keeping the client and server in sync can be difficult while developing a web app. That’s particularly true if you are using text-based data such as JSON in your API calls (though the flexibility and readability of JSON over binary data should not be disregarded).
JSON Schemas and Contract Tests
On a previous project that used Ember and Rails, I attempted to keep the server and client in sync by using JSON Schema and writing contract tests to verify that all data at all endpoints conformed to the Schema. This worked, but it was a significant maintenance pain. Additionally, unless you write the Schema in a very restrictive way, it is easy to update your code and neglect to update the Schema. Finally, in my opinion, writing ActiveModelSerializers feels pretty disjointed from the final shape of the JSON data that is serialized.
I’d rather convert to an intermediate data structure with obvious serialization rules (i.e., a structure that only contains numbers, strings, lists, and maps of strings to the aforementioned types). Contract tests also feel very boilerplate. So I left that project feeling mildly unsatisfied with my method of keeping the client and server in sync.
My current project is also using Ember for the front end, but it uses ASP.NET Web API (in C#) on the back end. In this project, there are more developers–and to some extent, more siloing–so I felt very strongly that we needed to know the server and the client were in sync on their API and data structures.
I looked at contract tests again, but as I said, they are annoying to maintain and contain lots of boilerplate. They wouldn’t be necessary at all if I could share just a bit of data structures and/or code between the front end and the back end.
Sharing Types and Endpoints
I got to thinking: The server-side code already defines all the types that will be serialized to JSON in easy-to-read, static types. Could I share them with the client in some way? Ember already has a standard way to define data structures (extending Ember.Object, or DS.Model if you are using Ember Data). Could I find the relevant data structures in the C# back end and programmatically convert and emit them as Ember Object types?
This seemed fairly simple, though after a fair bit of searching, I didn’t find an existing library that quite fit the bill. So I started to ponder how much work it would be to implement the conversion myself.
Ember model generator
As it turned out, a bit of reflection on the C# code actually made doing so pretty easy:
- Find all controllers (all types in an assembly that extend ApiController).
- Find all types for input and output on controller actions (MethodInfo instances), unwrapping IEnumerable<>, Task<> etc.
- Crawl the properties on all types from step 2 and include referenced types.
- Use (parts of) my JSON serialization library to emit the correct names for keys/properties in JSON/JavaScript.
- Finally, emit JavaScript code based on consuming JavaScript framework requirements–in this case, as
export const TypeName = Ember.Object.extend({...
Here’s an example conversion. Given this ASP.NET code:
public class BlogPost {
public string Author { get; set; }
public string Text { get; set; }
public int Likes { get; set; }
public Comment[] Comments { get; set; }
}
public class Comment {
public string UserName { get; set; }
public string Text { get; set; }
}
my generator creates the following Ember code:
export const Comment = DS.Model.extend({
userName: DS.attr('string'),
text: DS.attr('string')
});
export const BlogPost = DS.Model.extend({
author: DS.attr('string'),
text: DS.attr('string'),
likes: DS.attr('number'),
comments: DS.hasMany('comment')
});
API generation
At this point, I am able to generate models for use in JavaScript code that I know match the server–that’s where they came from, after all. Going a step further, as we are not using Ember Data, I also found it useful to export an api.js file that wraps the URLs and methods for each Controller Action endpoint. Essentially, this lets my calling code use something more semantically meaningful than $.ajax()
.
So for example, if I have two Controllers that use the models from the previous example:
[RoutePrefix("api/blog-posts")]
public class BlogPostsController : ApiController
{
[HttpGet]
[Route("")]
public async Task> GetPosts()
{
//...
}
[HttpPost]
[Route("")]
public async Task CreatePost(BlogPost data)
{
//...
}
[HttpDelete]
[Route("{id}")]
public async Task DeletePost(int id)
{
//...
}
}
[RoutePrefix("api/blog-posts/{blogId}/comments")]
public class BlogPostCommentsController : ApiController
{
[HttpPost]
[Route("")]
public async Task CreateComment(int blogId, BlogPostComment cmt)
{
//...
}
}
That is exported as follows for Ember:
blogPosts: {
getPosts: endPoint('GET', route`api/blog-posts`, {"returnDataType":["blog-post"]}),
createPost: endPoint('POST', route`api/blog-posts`, {"inputDataType":"blog-post","returnDataType":"blog-post"}),
deletePost: endPoint('DELETE', route`api/blog-posts/${0}`, {})
},
blogPostComments: {
createComment: endPoint('POST', route`api/blog-posts/${0}/comments, {"inputDataType":"blog-post-comment","returnDataType":"blog-post-comment"})
},
These API endpoints live in a service that can be accessed as follows:
export default Ember.Controller.extend({
api: Ember.inject.service(),
createPost(data) {
return this.get('api.blogPosts').createPost(data);
}
});
Input data types for each endpoint are checked to make sure they match the expected type (by comparing constructors), and return types are automatically deserialized from raw JSON/JavaScript objects into the model type used by my application. These types are specified by the generated “inputDataType” and “returnDataType” properties.
Making updates
With the generator in place, when new types and endpoints are added to the back end, all I have to do is run the tool and dump the output files in the right location on my Ember project. (I put them in app/generated/, to make sure they don’t collide with any other files.) Or, if you want the front end to lead the back end–as I’d recommend–you can make the required changes to the generated files, and then later on when the back end catches up, you just need to make sure the generated files don’t change when you re-run the generator.
Conclusion
I’m really happy with how this turned out. It didn’t take very long to implement, and now I know that both the back end and the front end will use the same data and the same endpoints. This technique has already helped me catch potential bugs early when changing data types. C# may be a bit verbose for my tastes, but I say, “If you’ve got a static type system, you should make good use of it.”
Where have you seen a little bit of tooling work go a long way on a project?
Really nice article. It surprises me the lack of tooling around this concept. So much friction involved in something that to me seems so fundamental. Thanks for posting.