• Ian Stuart's picture

    Fully Stacked

    Ian Stuart / August 10, 2018
  • When I started in this game, and I freely admit that was last century, "Full Stack" meant web servers and LAMP, and LAMP stood for Linux, Apache, Mysql, and PHP (or maybe Python).

    Now, to be fair, that misses CSS... but the point is still that "Full Stack" was pretty simple.

    Don't get me wrong - this "full stack" is still around & valid: your Joomla comemrce-site and your Wordpress Blogg, with their fancy front-ends & bespoke plugins is still essentially LAMP.

    When it comes to a "Full Stack" DevOps environment here at EDINA, the world has got way more complex:

    We could be talking "terraform" as a tool to create VMs in high-availabilities clusters - be they commercial Microsoft/Google/Amazon infrastructure, or an inhouse OpenShift offering. We then have to talk about "puppet" to manage the configuration of those hosts. Are we deploying services directly to VMs, as Docker containers on VMs, or even as Services in a cloud (Docker Swarm or Kubenetes?)

    How are we managing those VMs? Do we expect someone to log in and manually configure & update them, or use Puppet? Are we talking Puppet 3 or Puppet 4? Are we using R10K? Are we using Hieradata for variables? What about encoded data (we don't want passwords in plain text, now do we?) For code-development, lets ignore all the hip trends for developement cycle styles, and concentrate on the technologies we use: There's Git for content management; there's Jenkins or GitLab/CI for integration & deployment. There's Docker and/or Vagrant for creating local instances of whatever it is you're working on... and we haven't even talked about service structure yet!

    The idea of "monolothic" is old-skool - we want front-ends that talk to back-ends that talk to data-services... or the MVC framework1 (think Django for Python, Catalyst for Perl, Rails for Ruby.) We want to have predictive inputs and dynamic updates.

    Our data-sources may be on an SQL database, a full-text search server (such as solr), or a noSQL store (and there's a vast subject in it's own right! tuple-store v's object-store v's key-calue store v's ....)

    On top of our data-store, we create an API to take in [mostly http] requests, interact with the data-store(s), and return data in some pre-defined format. There's a niceity I picked up from (I think) Lincoln Uni called "Eat Your Own Dogfood" - which essentially means you should write your APIs to be usable by any service, not just your own.

    Finally we have our front-end, our UI, that thing that has a UIX (you gotta love how people want to make thing sound important) The front end is usually html, but are we talking xhtml-4, or html-5? Are we creating responsive designs that effortlessly adjust themselves for a mobile, a tablet, or a PC? User Interface design has moved on a lot since the 1990's: there's been vast swathes of research into colour, layout, presentation, shapes, & responsiveness... all those things we call "look and feel" - that's a complex web of HTML elements, and cascading style definitions. HTML & CCC is, in it's own right, a skill: creating a responsive design with the flexibility to work across multiple screen sizes, under multiple OS'n'browser combinations, yet remain maintainable and manageable.

    Yet UIs are, these days, much more than just HTML & CSS: we have predictive inputs (often called "Ajax"); dynamic content; and all sorts of client-side cleverness. This cleverness can be Javascript, or the Typescript superset. The UI experience can be assembled in an an application framework such as Angular (JS, 2, 4, 5, or 6)

    All development work needs testing, which brings us back to the use of Virtual Machines or Docker (which means docker-compose files) - and extensive test files. Testing - there's another art-form: knowing what to test; ensuring you have coverage for all the calls; ensuring you are covering your edge-cases; ensuring you're testing for failures as well as successes; and testing efficiently. Someone has to write all those tests too!

    Finally, there's peripheral things you need to know about: things like SQL, JSON, XML, YAML, HTTP requests... perhaps not programming technologies or part of any stack - but part of the information world.

    I counted it - covering just 2 of the services I work on, I have needed to know (as in directly use or write content for/in): Angular6, Catalyst, CSS, Django, Docker, Gitlab/CI, HTML, JQuery, JSON, Jupyter [Notebooks & hub], Kubenetes, Perl, Puppet, Python, Rancher, SQL, Template Toolkit, Terraform, TypeScript, XML, YAML

    [1] MVC is actually not new - it dates back to the mid 70s (that's 1970's)