Terraform Stateserver Using Python

$ mkdir tf-state-server
$ cd tf-state-server

$ virtualenv -p python3 venv
created virtual environment CPython3.8.6.final.0-64 in 174ms
  creator CPython3Posix(dest=/home/rrosso/tf-state-server/venv, clear=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/rrosso/.local/share/virtualenv)
    added seed packages: pip==20.1.1, pkg_resources==0.0.0, setuptools==44.0.0, wheel==0.34.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

$ source venv/bin/activate

(venv) $ pip install -U -r requirements.txt
Collecting flask
  Using cached Flask-1.1.2-py2.py3-none-any.whl (94 kB)
Collecting flask_restful
  Downloading Flask_RESTful-0.3.8-py2.py3-none-any.whl (25 kB)
Collecting itsdangerous>=0.24
  Using cached itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting Werkzeug>=0.15
  Using cached Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting Jinja2>=2.10.1
  Using cached Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
Collecting click>=5.1
  Using cached click-7.1.2-py2.py3-none-any.whl (82 kB)
Collecting pytz
  Downloading pytz-2021.1-py2.py3-none-any.whl (510 kB)
     |████████████████████████████████| 510 kB 3.0 MB/s
Collecting six>=1.3.0
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting aniso8601>=0.82
  Downloading aniso8601-9.0.1-py2.py3-none-any.whl (52 kB)
     |████████████████████████████████| 52 kB 524 kB/s
Collecting MarkupSafe>=0.23
  Using cached MarkupSafe-1.1.1-cp38-cp38-manylinux2010_x86_64.whl (32 kB)
Installing collected packages: itsdangerous, Werkzeug, MarkupSafe, Jinja2, click, flask, pytz, six, aniso8601, flask-restful
Successfully installed Jinja2-2.11.3 MarkupSafe-1.1.1 Werkzeug-1.0.1 aniso8601-9.0.1 click-7.1.2 flask-1.1.2 flask-restful-0.3.8 itsdangerous-1.1.0 pytz-2021.1 six-1.15.0

(venv) $ python3 stateserver.py
 * Serving Flask app "stateserver" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://192.168.1.235:5000/ (Press CTRL+C to quit)
...

➜  cat main.tf
terraform {
  backend "http" {
    address = "http://192.168.1.235:5000/terraform_state/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a"
    lock_address = "http://192.168.1.235:5000/terraform_lock/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a"
    lock_method = "PUT"
    unlock_address = "http://192.168.1.235:5000/terraform_lock/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a"
    unlock_method = "DELETE"
  }
}

➜  source ../env-vars

➜  terraform init

Initializing the backend...
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "http" backend. No existing state was found in the newly
  configured "http" backend. Do you want to copy this state to the new "http"
  backend? Enter "yes" to copy and "no" to start with an empty state.

  Enter a value: yes

Successfully configured the backend "http"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.oci: version = "~> 4.17"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

server shows
---
````bash
...
192.168.1.111 - - [16/Mar/2021 10:50:00] "POST /terraform_state/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a?ID=84916e49-1b44-1b32-2058-62f28e1e8ee7 HTTP/1.1" 200 -
192.168.1.111 - - [16/Mar/2021 10:50:00] "DELETE /terraform_lock/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a HTTP/1.1" 200 -
192.168.1.111 - - [16/Mar/2021 10:50:00] "GET /terraform_state/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a HTTP/1.1" 200 -
...

$ ls -la .stateserver/
total 24
drwxrwxr-x 2 rrosso rrosso 4096 Mar 16 10:50 .
drwxrwxr-x 4 rrosso rrosso 4096 Mar 16 10:43 ..
-rw-rw-r-- 1 rrosso rrosso 4407 Mar 16 10:50 4cdd0c76-d78b-11e9-9bea-db9cd8374f3a
-rw-rw-r-- 1 rrosso rrosso 5420 Mar 16 10:50 4cdd0c76-d78b-11e9-9bea-db9cd8374f3a.log

$ more .stateserver/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a*
::::::::::::::
.stateserver/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a
::::::::::::::
{
    "lineage": "9b756fb7-e41a-7cd6-d195-d794f377e7be",
    "outputs": {},
    "resources": [
{
"instances": [
{
"attributes": {
"compartment_id": null,
"id": "ObjectStorageNamespaceDataSource-0",
"namespace": "axwscg6apasa"
},
"schema_version": 0
}
],
...
    "serial": 0,
    "terraform_version": "0.12.28",
    "version": 4
}
::::::::::::::
.stateserver/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a.log
::::::::::::::
lock: {
    "Created": "2021-03-16T15:49:34.567178267Z",
    "ID": "7eea0d70-5f53-e475-041b-bcc393f4a92d",
    "Info": "",
    "Operation": "migration destination state",
    "Path": "",
    "Version": "0.12.28",
    "Who": "rrosso@desktop01"
}
unlock: {
    "Created": "2021-03-16T15:49:34.567178267Z",
    "ID": "7eea0d70-5f53-e475-041b-bcc393f4a92d",
    "Info": "",
    "Operation": "migration destination state",
    "Path": "",
    "Version": "0.12.28",
    "Who": "rrosso@desktop01"
}
lock: {
    "Created": "2021-03-16T15:49:45.760917508Z",
    "ID": "84916e49-1b44-1b32-2058-62f28e1e8ee7",
    "Info": "",
    "Operation": "migration destination state",
    "Path": "",
    "Version": "0.12.28",
    "Who": "rrosso@desktop01"
}
state_write: {
    "lineage": "9b756fb7-e41a-7cd6-d195-d794f377e7be",
    "outputs": {},
    "resources": [
{
"instances": [
{
"attributes": {
"compartment_id": null,
"id": "ObjectStorageNamespaceDataSource-0",
"namespace": "axwscg6apasa"
},
"schema_version": 0
}
...

...
192.168.1.111 - - [16/Mar/2021 10:54:12] "PUT /terraform_lock/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a HTTP/1.1" 200 -
192.168.1.111 - - [16/Mar/2021 10:54:12] "GET /terraform_state/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a HTTP/1.1" 200 -
192.168.1.111 - - [16/Mar/2021 10:54:15] "DELETE /terraform_lock/4cdd0c76-d78b-11e9-9bea-db9cd8374f3a HTTP/1.1" 200 -
...

$ more requirements.txt stateserver.py
::::::::::::::
requirements.txt
::::::::::::::
flask
flask_restful
::::::::::::::
stateserver.py
::::::::::::::
#!/usr/bin/python3

import flask
import flask_restful
import json
import logging
import os

app = flask.Flask(__name__)
api = flask_restful.Api(app)

@app.before_request
def log_request_info():
    headers = []
    for header in flask.request.headers:
        headers.append('%s = %s' % (header[0], header[1]))

    body = flask.request.get_data().decode('utf-8').split('\n')

    app.logger.debug(('%(method)s for %(url)s...\n'
                      '    Header -- %(headers)s\n'
                      '    Body -- %(body)s\n')
                     % {
        'method': flask.request.method,
        'url': flask.request.url,
        'headers': '\n    Header -- '.join(headers),
        'body': '\n           '.join(body)
    })

class Root(flask_restful.Resource):
    def get(self):
        resp = flask.Response(
            'Oh, hello',
            mimetype='text/html')
        resp.status_code = 200
        return resp

class StateStore(object):
    def __init__(self, path):
        self.path = path
        os.makedirs(self.path, exist_ok=True)

    def _log(self, id, op, data):
        log_file = os.path.join(self.path, id) + '.log'
        with open(log_file, 'a') as f:
            f.write('%s: %s\n' %(op, data))

    def get(self, id):
        file = os.path.join(self.path, id)
        if os.path.exists(file):
            with open(file) as f:
                d = f.read()
                self._log(id, 'state_read', {})
                return json.loads(d)
        return None

    def put(self, id, info):
        file = os.path.join(self.path, id)
        data = json.dumps(info, indent=4, sort_keys=True)
        with open(file, 'w') as f:
            f.write(data)
            self._log(id, 'state_write', data)

    def lock(self, id, info):
        # NOTE(mikal): this is racy, but just a demo
        lock_file = os.path.join(self.path, id) + '.lock'
        if os.path.exists(lock_file):
            # If the lock exists, it should be a JSON dump of information about
            # the lock holder
            with open(lock_file) as f:
                l = json.loads(f.read())
            return False, l

        data = json.dumps(info, indent=4, sort_keys=True)
        with open(lock_file, 'w') as f:
            f.write(data)
        self._log(id, 'lock', data)
        return True, {}

    def unlock(self, id, info):
        lock_file = os.path.join(self.path, id) + '.lock'
        if os.path.exists(lock_file):
            os.unlink(lock_file)
            self._log(id, 'unlock', json.dumps(info, indent=4, sort_keys=True))
            return True
        return False

state = StateStore('.stateserver')

class TerraformState(flask_restful.Resource):
    def get(self, tf_id):
        s = state.get(tf_id)
        if not s:
            flask.abort(404)
        return s

    def post(self, tf_id):
        print(flask.request.form)
        s = state.put(tf_id, flask.request.json)
        return {}

class TerraformLock(flask_restful.Resource):
    def put(self, tf_id):
        success, info = state.lock(tf_id, flask.request.json)
        if not success:
            flask.abort(423, info)
        return info

    def delete(self, tf_id):
        if not state.unlock(tf_id, flask.request.json):
            flask.abort(404)
        return {}

api.add_resource(Root, '/')
api.add_resource(TerraformState, '/terraform_state/<tf_id>')
api.add_resource(TerraformLock, '/terraform_lock/<tf_id>')

if __name__ == '__main__':
    # Note this is not run with the flask task runner...
    app.log = logging.getLogger('werkzeug')
    app.log.setLevel(logging.DEBUG)
    #app.run(host='0.0.0.0', debug=True)</tf_id></tf_id>