Rapid Application Prototyping Framework

One thing that is clear to me, having worked in HighQ support now for approx. 3 years, is that there are a lot of people in our client's organisations, who have a great foundational knowledge of the of the web (HTML, CSS, JavaScript), despite not necessarily being developers per se. Obviously that's great, but all too often consuming a REST API is something which they may not have done before, and it can quickly become a point of frustration. I fully sympathize with this, having undergone a similar journey myself. As a web-savvy person in a law firm, you probably want to be able to populate web pages with data from HighQ and there may not be a clear path as to how to achieve this. Further to this, the overhead of authenticating and managing tokens, etc, as well as stepping through all of the fiddly bits of constructing HTTP requests can seem overwhelming at first.

These types of user may also be writing their code within a restricted environment such as our own rich-text editor or a 'no-code' platform. Our API doesn't always play nicely when being called from client side code and no-or-low code apps don't always support the OAuth2 workflow fully. 

This got me thinking; is there a better way to get these types of web-curious users up and running more quickly?

The trend in web development frameworks of late has decidedly been towards 'micro' frameworks, which pertain to be non-opinionated and give the developer as much flexibility as possible. But what if one was to design a web development framework for non-developers? 'Magicals' suddenly become a good thing. In this case,  the proposed framework could abstract away a lot of the grubbing around required to get an integration with a HighQ system up and running, leaving your legal technologists, etc as much freedom as possible to do what they actually want to be doing, which is creating informative and beautiful web pages using the web technologies that they are familiar with. 

This lead me to create a Python base micro-framework which I've dubbed 'HighQ Server Pages' or 'HQSP' . Python is a strategic choice here as its a high level, dynamically typed and interpreted language, meaning its considered quicker and easier to learn than a lot of others, (its detractors even refer to it as 'executable psuedo-code', which I think is actually a good thing!). A user with decent working knowledge of JavaScript should be able to pick up its basics without issue. In the world of finance , Bank Of America and JP Morgan have each made a similar investment in Python because, amongst other things, Developer onboarding is more rapid. Python is used heavily in their in-house Application Platforms. 

HighQ Server Pages only supports 'function' based controllers for these reasons. Its easy to understand a Python function. Anyone developing an application in the framework, doesn't have to wrap their head around Object-Oriented Programming. A function simply receives and request and returns a response... that's it! This is the W3 Schools Python function example:

def my_function(country):
  print('I am from ' + country)

my_function('Wales')

$ 'I am from Wales'

In the world of the web, it doesn't take a big leap of the imagination to apply this to the request/response cycle: 

from framework import HttpResponse

def foo(request):
    return HttpResponse('Bar!')

Here is an example app built with HQSP!
 

I have downloaded the Bootstrap framework, with the minified asset bundles, like so, along with an html template. Static resources live underneath the 'Static' directory. HTML Templates live under the 'Templates' directory. To labour an earlier point, there is no flexibility here. In terms of organising these files, HQSP is rigid and opinionated, so that its users are concentrating on the 'building the App' part of the process, rather than something like 'overriding the default path for static files'.
 

static
|--css
   |--bootstrap.min.css
|--js
   |--bootstrap.min.css

templates
|--sites.html

In the same top level directory I have a settings.ini file, which effectively just captures the details of an initial OAuth token call. 

[AUTH]
access_token = REDACTED_your_token_here
access_expiry = 2592000
refresh_token= REDACTED_your_token_here
refresh_expiry = 2628000
token_type = bearer
user = peter.simpson@highq.com

[INSTANCE]
host = https://support.highq.com/support

My demo.py file (where the user defined Application actually lives) looks like this: 

from hqsp import Application
import requests as http_handler

app = Application()

@app.set_headers
def highq_site(headers, base_url, request, response, site_id):

    external_response = http_handler.get(
        f"{base_url}/api/13/sites/{site_id}", headers=headers
    )

    response.body = app.template("sites.html", context=external_response.json())



app.add_routes({"/site/{site_id}": highq_site})

The developer declares an Application object (which is an instance of a HighQ Server Pages Application), and then they are able to use a decorator to 'wrap' various functions, which magically passes in everything needed to make the API call to the Collaborate RESTful service. You will then see that the response's body is set to be the application's templating method which loads together the html file and a JSON structure of the data returned from the API call. Under the hood, the Framework is leveraging the popular Jinja2 templating language. 

My sites.html file looks like so: 

<html>
<header>
    <title>{{ sitename }}</title>
<!-- loading in the static assets, scroll down for js --> 
    <link href="/css/bootstrap.min.css" type="text/css" rel="stylesheet">
</header>
<body>

<!-- skipping over the navbar for brevity --> 

<main class="container">
  <div class="bg-light p-5 rounded">
    <h1>{{sitename}}</h1>
        <table class="table table-hover">
          <thead class="thead-dark">
            <tr>
              <th scope="col">Enabled Modules</th>
            </tr>
          </thead>
          <tbody>
          {% for key in module %}
              {% set module_name = key %}
               {% if module[module_name]['enable'] == '1' %}
                    <tr>
                      <td>{{ module_name }}</td>
                    </tr>
              {% endif %}
            {% endfor %}
          </tbody>
        </table>
  </div>
</main>

<script src="/js/bootstrap.min.js"></script>
</body>
</html>

You can see that we are using Jinja2 to pull out values from the data returned by the HighQ API, as well as looping over the modules and displaying them if they are enabled. 

The last thing to mention from the demo.py file is the 'add_routes' method. This creates a mapping from a path to a function and you'll notice that there is a path parameter in there {site_id}, so the page will load different data depending on the value that gets passed to it. 

HQSP conforms to the WSGI standard for Python web frameworks, so it can be run by any WSGI Server. I choose the popular Gunicorn, which means the Application can be run like so: 
 

$ gunicorn demo:app

Here are the results: 

Hardly very visually impressive, but hopefully the intent is clear. You can now write a simple Python function to fetch data from the HighQ backend and serve it into an HTML template, along with any stylings and JavaScript required. A much more complex application could easily be extended from this core set of tools. 

Here is another screenshot just to prove its actually dynamic!

The framework also takes care of token refreshment, but a user purposefully doesn't know anything about that. So long as the function controllers are decorated with '@app.set_headers', behind the scences: HQSP works out whether or not a refresh is required and persists the tokens to storage. 

All in all, the actual HQSP framework is less than a few hundred lines of code. Its mostly just gluing together pre-existing open source tools in the Python world. These tend to be packages for dealing with a single well trodden Web programming problem. WebOb provides the Http Objects, Jinja2 (already mentioned) does the templating, WhiteNoise serves static files. If I built this framework out beyond a proof of concept, perhaps I would roll my own for some or all of these components? I have also made 0 security considerations, hence it being a self-declared 'prototyping' framework. These Applications are intended only to be served locally. Once its users have their POCs, the idea would currently be that they would hand over to their developers, who could then build the ideas out in the MVC framework of their choice. 

Web Frameworks are plentiful and essentially represent a problem already solved. The only real reason to build one would be to either learn how they work (which is a valuable exersize in and of itself), or if you have a use-case so specific that it can be better served with a refinement of an already existing paradigm. I hope that the notion of a framework for non developers wanting to leverage a particular API, is clear. If you don't build out an entire framework for how you interact with our API, some kind of reusable library or client can hopefully lead to a more pleasant developer experience, if you regularly work with our services.