Solving the (my) pains when working with single page applications
25 Sep 2015Lately, my personal trend around working with a single page application has
been to create two separate projects. One called client and the other called
server.
I discussed the up/downs in this StackOverflow answer. @elado also answered it and emphasized how he works in this answer.
If you’ve read the StackOverflow question I linked to, you can understand that
when working like this, you have a set of challenges and some
misunderstandings, I think some of them are
worth mentioning here.
Assets
When working with Rails, you have the asset pre-compile stage, called the asset pipeline. Now, this is a source of hate and controversy in the rails community,
I definitely do not want to open that Pandora’s box, BUT, it raises a legitimate question. If I have a client application and a server API, who is in charge of assets?
When you get into the complexity of managing assets in a production
environment, you understand why it’s important. In order to deploy assets you
will need to compile them, sign them for a release and put them where servers
can easily access them. That goes to CSS and JS.
In development, you would want to work with as many files as you can,
modularizing them, but in production (at least for HTTP/1.1) you would ideally
want the browser to link to as few as possible.
Configuration
Where’s my API?
Locally, it’s located at http://localhost:300, staging is stg.your-app.com,
etc… Normally, you would want to inject those settings during deployment
right?
I discussed this with @elado
This will make most developers go like:
Now, if you modularize your app correctly, you should not have a lot of places
where you inject the Config factory, but if not, you are in for a world of
hurt.
Tools
I remember the joys of working with a monolithic rails application, everything
was either rails g or some rake that you could easily find with rake -T.
Now that you have a client and an API, you are working with more tools, each of
them brings challenges to the table that you need to deal with.
Scratching the surface
This is just scratching the surface of the challenger, when you go into running
these things to production you have many more
Solving the problems
All of the challenges and problems are solvable, I would like to discuss how I
personally solved some of them in the next part
API location
THE best solution I have for the API location, one that is scalable in
production as well is to not use an api location.
Instead, have your app call /api, and handle it on the server. Lets see how
that works on development first and then on the server.
When I run grunt serve and rails server I essentially have two
applications. One on localhost:9000 and the other on localhost:3000.
I created a dead simple Go proxy, this opens a server on
http://localhost:5050. When I ask for a URL that start with /api it will
proxy the request to localhost:3000, everything else goes to
localhost:9000.
Here’s a basic diagram to show how this works:
This solves a few problems.
First One is security configuration called CORS. I discussed handling Cors with Angular before.
Second, This will allow you to deploy this anywhere and the server it’s being
deployed to will control the state of whether this application is
production/staging or something else.
Nginx is awesome for those kind of configuration, here’s a snippet from our
configuration
server {
client_max_body_size 30M;
listen 80;
... REDACTED
server_tokens off;
root "<%= @path %>/current/public";
location /api {
try_files $uri @gogobot-backend;
}
location @gogobot-backend {
...REDACTED
}
}
Note: This template is from chef, this gets evaluated by Ruby before going to
the server, hence the <%= @path %> code…
Now, this means that the client side code and the server side code need to
“live” on the same server, but this is definitely not a restriction you need to
have. For us it makes sense, for you it might make more sense to have a single
server for the client and multiple servers for the backend and have Nginx proxy
to the load balancer. You are in control here.
Assets
I use bower and grunt in order to manage assets. I do not manage any of the
assets in Rails.
Here’s what it looks like in the code
<!-- build:js(.) scripts/vendor.js -->
<!-- bower:js -->
<script src="bower_components/jquery/dist/jquery.js"></script>
<script src="bower_components/angular/angular.js"></script>
<script src="bower_components/angular-animate/angular-animate.js"></script>
<script src="bower_components/angular-aria/angular-aria.js"></script>
<script src="bower_components/angular-cookies/angular-cookies.js"></script>
<script src="bower_components/lodash/lodash.js"></script>
<script src="bower_components/angular-google-maps/dist/angular-google-maps.js"></script>
<script src="bower_components/angular-messages/angular-messages.js"></script>
<script src="bower_components/angular-resource/angular-resource.js"></script>
<script src="bower_components/angular-route/angular-route.js"></script>
<script src="bower_components/angular-sanitize/angular-sanitize.js"></script>
<script src="bower_components/angular-touch/angular-touch.js"></script>
<script src="bower_components/bootstrap-sass-official/assets/javascripts/bootstrap.js"></script>
<!-- endbower -->
<!-- endbuild -->
<!-- build:js({.tmp,app}) scripts/scripts.js -->
<script src="scripts/app.js"></script>
<script src="scripts/controllers/application.js"></script>
<script src="scripts/components/field/gg_field_controller.js"></script>
<script src="scripts/components/field/gg_field.js"></script>
<script src="scripts/controllers/main.js"></script>
<script src="scripts/controllers/login.js"></script>
<!-- endbuild -->
The default Yeoman Angular generator does the trick, use it in order to start
your applications and never look back.
CDN friendliness
For us at Gogobot, we have 3 levels of caching, I will not go too much into
details here, but the first level of cache is the CDN, we simply cache the page
as it is.
When you have a client/server application, this provides a few challenges as
well. Here’s what we do.
Obviously, we have a random key after each file when we build, this is handled
by grunt (the code above, reference the Yeoman generator for more details).
The perceptive of you will see a challenge here as well, since you have a JS
file but the HTML file actually might reference something different, this file
must be on the server in order for the HTML to work
The solution
The solution to that is also a bit complex, lets discuss it one by one. First,
lets make sure we have the files on the servers when we need them.
The solution for this is actually pretty straight forward.
We keep all the JS and css files on the servers, we don’t delete them with
every deployment. Instead, in a shared folder we have
/v/{version}/file.{version}.js/css. This way, when the server asks for a
file, we proxy that into the right folder. (using Nginx as well).
This is actually how our main rails app works right now as well, since we cache
the pages as a whole, we want to serve CSS for the page even if the page itself
is stale (pages are cached for 24-48 hours). You can go to This
page and have a look at the source, find the
CSS files.
Here’s an example code from our deployment script that cleans up this library
assets_path = "#{latest_release}/public/v"
dirs = capture("ls -tr #{assets_path}", hosts: server.host).split("\n")
trim = dirs.size - 40
if trim > 0
logger.info "Removing #{trim} assets for release older than 40 versions..."
dirs[0..trim].each { |d| run "rm -rf #{assets_path}/#{d}", hosts: server.host }
else
logger.info "Not removing old releases, we have only #{dirs.size} out of 40 stored."
end
The second problem is harder to solve, and that’s: What happens if an old JS
tries to call an API that is no longer there, or responds in a manner that you
can’t predict anymore?
For this problems there are many solutions as well, you can go as crazy as
versioning and migrations for an API or you can version your API smartly for
breaking changes.
We chose the latter, we simply version our API in a way where we don’t remove
fields or change a field type unless you “release” a new version. Since our
API’s server the mobile application as well, those need to remain stable for
users and cannot break.
Choose how far back you want to support and go along with it.
Development against a server
From my experience in the past, this is definitely more noticeable for
developers that don’t use TDD, but, it can be a pain for ones that do as well.
When you work with a client application and the server is developed separately,
there’s often a gap between how people work, the client side developer might
need some response from the server that isn’t there yet. It could be a field,
it could be an endpoint that doesn’t even exist yet.
Facebook’s been doing amazing work with GraphQL, this means that the client
describes the response it needs from the server, the wrote a good post about it
here: GraphQL Introduction
However, for most applications you will have a predefined API endpoint that
sends back a predefined JSON response.
If you are starting from scratch right now, definitely look into customizing
this, make it easier for the client to define the server response, but if you
don’t, the solution for you is a proxy-stub server
There are so many open source solutions for this I can’t even count, Janus is one of them.
You can configure endpoints and API response and it will send you back the
stubbed response.
This is awesome if you want to develop visually, style a web page and what not,
all of this without compromising your progress by other teams.
Closing thoughts
While writing this post, I got the same feeling you get (if you got to this
point) probably. Is this all worth it? The answer is yes, this is what the
internet looks like these days, a client->api.
When you have an API you can serve all clients by the same logic, it’s actually
a great way to make sure you have less bugs and deliver more stable code.
Feel free to discuss in the comments, if you have any questions/comments, I
would love to read them.