首頁>技術>

之前有過朋友問我Flask、Express這些框架是如何在函式中執行,他是怎麼樣的一個機制?還有人問我如何做一個Component?看了一下騰訊雲Serverless架構現在支援的框架:

我發現雖然支援了很多,但是我比較鍾愛的Django貌似沒有,正好想到了部分人的疑惑,所以在這裡,我就簡單的和大家說一下,我如何做一個Django的Component。

首先第一步,我們要知道其他的框架是怎麼執行的,例如Flask等,我們先通過騰訊雲的Flask-Component,按照他的說明部署一下:

非常簡單輕鬆愉快的部署上線,然後在函式的控制檯,我們把部署好的下載下來,研究一下:

下載解壓之後,我們可以看這樣一個目錄結構:

藍色框起來的,是依賴包,黃色的app.py是我們的自己寫的程式碼,那麼紅色圈起來的是什麼?這兩個檔案從哪裡出來的?

api_server.py檔案內容:

import app # Replace with your actual applicationimport severless_wsgi​# If you need to send additional content types as text, add then directly# to the whitelist:## serverless_wsgi.TEXT_MIME_TYPES.append("application/custom+json")​def handler(event, context): return severless_wsgi.handle_request(app.app, event, context)

可以看到,這裡面是將我們建立的app.py檔案引入,並且拿到了app這個物件,並且將event和context同時傳遞給severless_wsgi.py中的handle_reques方法中,那麼問題來了,這個方法是什麼?

這個方法內容好多......看著有點眼暈,但是,我們可以直接發現這一段程式碼:

這一段是什麼呢?這一段實際上就是將我們拿到的引數(event和context)進行轉換,轉換之後統一environ中,然後接下來通過werkzeug這個依賴,將這個內容變成request物件,並且與我們剛才說的app物件一起呼叫from_app方法。獲得到反饋:

並且按照API閘道器的響應整合的格式,將結果返回。

此時此刻,各位看官可能有點想法了,貌似有一丟丟靈感出現了,那麼我們不妨看一下Flask/Django這些框架的實現原理:

通過這個簡版的原理圖,和我剛才說的內容,我們可以想到,實際上正常用的時候要通過web_server,進入到下一個環節,而我們雲函式更多是一個函式,本不需要啟動web server,所以我們就可以直接呼叫wsgi_app這個方法,其中這裡的environ就是我們剛才的通過對event/context等進行處理後的物件,start_response可以認為是我們的一種特殊的資料結構,例如我們的response結構形態等。所以,如果我們自己想要實現這個過程,不使用騰訊雲flask-component,可以這樣做:

import sys​try: from urllib import urlencodeexcept ImportError: from urllib.parse import urlencode​from flask import Flask​try: from cStringIO import StringIOexcept ImportError: try: from StringIO import StringIO except ImportError: from io import StringIO​from werkzeug.wrappers import BaseRequest​__version__ = '0.0.4'​​def make_environ(event): environ = {} for hdr_name, hdr_value in event['headers'].items(): hdr_name = hdr_name.replace('-', '_').upper() if hdr_name in ['CONTENT_TYPE', 'CONTENT_LENGTH']: environ[hdr_name] = hdr_value continue​ http_hdr_name = 'HTTP_%s' % hdr_name environ[http_hdr_name] = hdr_value​ apigateway_qs = event['queryStringParameters'] request_qs = event['queryString'] qs = apigateway_qs.copy() qs.update(request_qs)​ body = '' if 'body' in event: body = event['body']​ environ['REQUEST_METHOD'] = event['httpMethod'] environ['PATH_INFO'] = event['path'] environ['QUERY_STRING'] = urlencode(qs) if qs else '' environ['REMOTE_ADDR'] = 80 environ['HOST'] = event['headers']['host'] environ['SCRIPT_NAME'] = '' environ['SERVER_PORT'] = 80 environ['SERVER_PROTOCOL'] = 'HTTP/1.1' environ['CONTENT_LENGTH'] = str(len(body)) environ['wsgi.url_scheme'] = '' environ['wsgi.input'] = StringIO(body) environ['wsgi.version'] = (1, 0) environ['wsgi.errors'] = sys.stderr environ['wsgi.multithread'] = False environ['wsgi.run_once'] = True environ['wsgi.multiprocess'] = False​ BaseRequest(environ)​ return environ​​class LambdaResponse(object): def __init__(self): self.status = None self.response_headers = None​ def start_response(self, status, response_headers, exc_info=None): self.status = int(status[:3]) self.response_headers = dict(response_headers)​​class FlaskLambda(Flask): def __call__(self, event, context): if 'httpMethod' not in event: print('httpMethod not in event') return super(FlaskLambda, self).__call__(event, context)​ response = LambdaResponse()​ body = next(self.wsgi_app( make_environ(event), response.start_response ))​ return { 'statusCode': response.status, 'headers': response.response_headers, 'body': body }

這樣一個流程,就會變得更加簡單,清楚。整個實現過程,可以認為是對web server部分進行了一種“截斷”或者是“替換”:

這就是對Flask-Component的基本分析思路,那麼按照這個思路,我們是否可以將Django框架部署上Serverless架構呢?那麼Flask和Django有什麼區別呢?我這裡的區別特指的是在執行啟動過程中。

仔細想一下,貌似並沒有區別,那麼我們是不是可以直接用Flask這個轉換邏輯,將flask的app替換成django的app呢?

把:

from flask import Flaskapp = Flask(__name__)

替換成:

import osfrom django.core.wsgi import get_wsgi_applicationos.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mydjango.settings')application = get_wsgi_application()

是否就能解決問題呢?

我們不妨試一下:

建立好Django專案,直接增加index.py:

# -*- coding: utf-8 -*-​import osimport sysimport base64from werkzeug.datastructures import Headers, MultiDictfrom werkzeug.wrappers import Responsefrom werkzeug.urls import url_encode, url_unquotefrom werkzeug.http import HTTP_STATUS_CODESfrom werkzeug._compat import BytesIO, string_types, to_bytes, wsgi_encoding_danceimport mydjango.wsgi​TEXT_MIME_TYPES = [ "application/json", "application/javascript", "application/xml", "application/vnd.api+json", "image/svg+xml",]​​def all_casings(input_string): if not input_string: yield "" else: first = input_string[:1] if first.lower() == first.upper(): for sub_casing in all_casings(input_string[1:]): yield first + sub_casing else: for sub_casing in all_casings(input_string[1:]): yield first.lower() + sub_casing yield first.upper() + sub_casing​​def split_headers(headers): """ If there are multiple occurrences of headers, create case-mutated variations in order to pass them through APIGW. This is a hack that's currently needed. See: https://github.com/logandk/serverless-wsgi/issues/11 Source: https://github.com/Miserlou/Zappa/blob/master/zappa/middleware.py """ new_headers = {}​ for key in headers.keys(): values = headers.get_all(key) if len(values) > 1: for value, casing in zip(values, all_casings(key)): new_headers[casing] = value elif len(values) == 1: new_headers[key] = values[0]​ return new_headers​​def group_headers(headers): new_headers = {}​ for key in headers.keys(): new_headers[key] = headers.get_all(key)​ return new_headers​​def encode_query_string(event): multi = event.get(u"multiValueQueryStringParameters") if multi: return url_encode(MultiDict((i, j) for i in multi for j in multi[i])) else: return url_encode(event.get(u"queryString") or {})​​def handle_request(application, event, context):​ if u"multiValueHeaders" in event: headers = Headers(event["multiValueHeaders"]) else: headers = Headers(event["headers"])​ strip_stage_path = os.environ.get("STRIP_STAGE_PATH", "").lower().strip() in [ "yes", "y", "true", "t", "1", ] if u"apigw.tencentcs.com" in headers.get(u"Host", u"") and not strip_stage_path: script_name = "/{}".format(event["requestContext"].get(u"stage", "")) else: script_name = ""​ path_info = event["path"] base_path = os.environ.get("API_GATEWAY_BASE_PATH") if base_path: script_name = "/" + base_path​ if path_info.startswith(script_name): path_info = path_info[len(script_name) :] or "/"​ if u"body" in event: body = event[u"body"] or "" else: body = ""​ if event.get("isBase64Encoded", False): body = base64.b64decode(body) if isinstance(body, string_types): body = to_bytes(body, charset="utf-8")​ environ = { "CONTENT_LENGTH": str(len(body)), "CONTENT_TYPE": headers.get(u"Content-Type", ""), "PATH_INFO": url_unquote(path_info), "QUERY_STRING": encode_query_string(event), "REMOTE_ADDR": event["requestContext"] .get(u"identity", {}) .get(u"sourceIp", ""), "REMOTE_USER": event["requestContext"] .get(u"authorizer", {}) .get(u"principalId", ""), "REQUEST_METHOD": event["httpMethod"], "SCRIPT_NAME": script_name, "SERVER_NAME": headers.get(u"Host", "lambda"), "SERVER_PORT": headers.get(u"X-Forwarded-Port", "80"), "SERVER_PROTOCOL": "HTTP/1.1", "wsgi.errors": sys.stderr, "wsgi.input": BytesIO(body), "wsgi.multiprocess": False, "wsgi.multithread": False, "wsgi.run_once": False, "wsgi.url_scheme": headers.get(u"X-Forwarded-Proto", "http"), "wsgi.version": (1, 0), "serverless.authorizer": event["requestContext"].get(u"authorizer"), "serverless.event": event, "serverless.context": context, # TODO: Deprecate the following entries, as they do not comply with the WSGI # spec. For custom variables, the spec says: # # Finally, the environ dictionary may also contain server-defined variables. # These variables should be named using only lower-case letters, numbers, dots, # and underscores, and should be prefixed with a name that is unique to the # defining server or gateway. "API_GATEWAY_AUTHORIZER": event["requestContext"].get(u"authorizer"), "event": event, "context": context, }​ for key, value in environ.items(): if isinstance(value, string_types): environ[key] = wsgi_encoding_dance(value)​ for key, value in headers.items(): key = "HTTP_" + key.upper().replace("-", "_") if key not in ("HTTP_CONTENT_TYPE", "HTTP_CONTENT_LENGTH"): environ[key] = value​ response = Response.from_app(application, environ)​ returndict = {u"statusCode": response.status_code}​ if u"multiValueHeaders" in event: returndict["multiValueHeaders"] = group_headers(response.headers) else: returndict["headers"] = split_headers(response.headers)​ if event.get("requestContext").get("elb"): # If the request comes from ALB we need to add a status description returndict["statusDescription"] = u"%d %s" % ( response.status_code, HTTP_STATUS_CODES[response.status_code], )​ if response.data: mimetype = response.mimetype or "text/plain" if ( mimetype.startswith("text/") or mimetype in TEXT_MIME_TYPES ) and not response.headers.get("Content-Encoding", ""): returndict["body"] = response.get_data(as_text=True) returndict["isBase64Encoded"] = False else: returndict["body"] = base64.b64encode(response.data).decode("utf-8") returndict["isBase64Encoded"] = True​ return returndict​​​def main_handler(event, context): return handle_request(mydjango.wsgi.application, event, context)

然後我們部署到函式上,看一下效果:

函式資訊:

from django.shortcuts import renderfrom django.http import HttpResponsefrom django.views.decorators.csrf import csrf_exempt​# Create your views here.@csrf_exemptdef hello(request): if request.method == "POST": return HttpResponse("Hello world ! " + request.POST.get("name")) if request.method == "GET": return HttpResponse("Hello world ! " + request.GET.get("name"))

通過部署完成,並繫結apigw觸發器,然後在postman中進行測試:

get:

post:

可以看到,通過我們對執行原理的基本剖析和對django的改造,我們已經通過增加一個檔案和相關依賴的方法,實現了Django上Serverless的過程。

接下來,我們看一下,如何將這個程式碼寫成一個Component:

首先Clone下來Flask-Component的程式碼:

然後,我們按照Django的部分模式進行修改:

第一部分,是我們可能會依賴的一個依賴包,以及我們剛才放入的index.py檔案。在使用者呼叫這個Component的時候,我們會把這兩個檔案,放入使用者的程式碼中,一併上傳。

第二部分是Serverless.js部分,這裡的一個基本格式:

const { Component } = require('@serverless/core')class TencentDjango extends Component { async default(inputs = {}) { } async remove(inputs = {}) { }}module.exports = TencentDjango

使用者在執行sls的時候,會預設呼叫default的方法,在執行sls remove的時候會呼叫remove的方法,所以可以認default的內容是部署,而remove的內容是移除。

部署這裡主要流程也蠻簡單的,首先將檔案進行復制和處理,然後直接呼叫雲函式的元件,通過函式中的include引數將這些檔案額外加入,再通過呼叫apigw的元件來進閘道器的管理,而使用者寫的yaml中inpust的內容,會在inputs中獲取,我們要做的就是對應的傳給不同的元件:

當然除了這兩部分對應放過去,上面的region等一些資訊也要對應的進行處理。而呼叫底層元件方法也很簡單:

const tencentCloudFunction = await this.load('@serverless/tencent-scf'const tencentCloudFunctionOutputs = await tencentCloudFunction(inputs)

處理好這裡之後,只需要修改一下package.json和readme就可以了。

目前,我已經完成了開源:https://github.com/gosls/tencent-django

也在NPM上進行了釋出:https://www.npmjs.com/package/@gosls/tencent-django

在使用的時候,只需要引入這個Component就好:

DjangoTest:  component: '@serverless/tencent-django'  inputs:    region: ap-guangzhou    functionName: DjangoFunctionTest    djangoProjectName: mydjango    code: ./    functionConf:      timeout: 10      memorySize: 256      environment:        variables:          TEST: vale      vpcConfig:        subnetId: ''        vpcId: ''    apigatewayConf:      protocols:        - http      environment: release

至此,完成了Django Component的開發和測試。

最新評論
  • BSA-TRITC(10mg/ml) TRITC-BSA 牛血清白蛋白改性標記羅丹明
  • 「喵咪開源軟體推薦(3)」全球IP庫-GeoLite2-City