服务
关于
CloudProse博客

At Trek10, we always try to consider the need for automation and repeatability with everything that we do. That’s why we focus on using tools like CloudFormation, 无服务器, and 江苏体彩, as well as building other tools. Recently, I was tasked with doing various maintenance tasks on a number of internal tools/projects. Some needed upgrades from Node.js 0.10, some needed 码 fixes, and most needed 江苏体彩. Today, we’re just going to focus on the 江苏体彩 part.

In spite of my past experience with Jenkins and TeamCity and our team’s experience with AWS (CodePipeline/CodeDeploy), I chose GitLab 江苏体彩 to standardize these projects. The biggest reason for this choice is history. As a project evolves, its 江苏体彩 configuration may change. If you ever need to go back in time, you may have difficulty deploying again. Since GitLab 江苏体彩 is based on a .gitlab-ci.yml config file that is committed with the 码, as long as a commit built and deployed then, it stands a 预先tty good chance of building and deploying now. Being able to tweak 江苏体彩 without leaving my editor was an additional bonus.

GitLab 江苏体彩中的速成课程

GitLab 江苏体彩 relies on having a .gitlab-ci.yml file in the root of your repo. 江苏体彩 for each commit is run against the .gitlab-ci.yml that is current for that commit. The fundamental unit of 江苏体彩 for GitLab is a “job”. A job is a construct that runs a bash script against a commit in a particular context. You might have one job to run tests, other jobs to build for staging or production, and other jobs to deploy to particular environments. In the config file, jobs are represented by top level maps (aka “objects”) that are not otherwise “reserved” GitLab 江苏体彩 maps. Examples of reserved top level maps: image (Docker image in which your jobs run), services (other Docker images that need to run while your jobs run), before_script (runs before every script), after_script (runs after every script), stages (redefines the stage names and order), variables (variables available to 所有 jobs), and cache (controls what is cached between 江苏体彩 runs; good for stuff from your package manager).

Every job 必须 belong to a stage (if left out, test is the default). Stages are run in a sequence, and 所有 of the jobs in a stage run with max parallelism available. The default stage sequence is: build, test, deploy. Each job also has before_script, after_script, variables, and cache. Defining these at a job level will override the top level configuration. The most important of these is variables, because your variables are what make the production deploy job’s context different from the staging deploy job’s context. variables is just a map with a bunch of key value pairs. Variables are consumed with a syntax simiar to bash: ${myVar}. There are some limitations that you should know:

  • 变量不支持bash变量扩展,替换,默认值等。
  • 变量不会递归或具有评估顺序,但是可以在作业级别变量中使用顶级变量。请参阅以下示例:

<pre><code>#您无法执行此操作(在同一地图中引用同级变量)变量:PROD_STAGE_NAME:产品PROD_URL: //thisismywebsite.com/${... 最后一个例子给了我们很多力量。我们一定会滥用它。

如前所述,作业在上下文中运行bash脚本。所以每项工作 必须 have a script. The last big thing that you need is “flow control”. By default, a job will run on every commit. Using the only, except, and when keys 所有ows you to control how jobs are triggered. only and except accept the following options:

  • Branch names, e.g. master or develop
  • 标签名称
  • JS样式RegExp文字可根据分支/标签名称进行评估
  • These special keywords: api, branches, external, tags, pushes, schedules, triggers, and web
    • Using branches and tags with only cause a job to be run for every branch or tag, respectively
  • 回购路径过滤器以处理回购分叉

一个更重要的事实:以句点字符开头的作业被禁用,例如: .my_disabled_job

那应该足以让我们开始。您可以找到更多的GitLab 江苏体彩文档 这里. The most useful bit is the .gitlab-ci.yml reference found 这里.

As with any new tool, I got to read and re-read the documentation and make some mistakes getting things right. By the time I was knee deep in this, I realized there was a need to 预先vent anyone from having to do this again, myself included. The solution requires 2 things: a well-designed 江苏体彩 template and a way to get that template into 所有 of your new repositories. Let’s tackle template design next.

设计模板

This part is hard to talk about in a completely generic manner. Instead, let’s walk through our use case. Looking at our projects past and 预先sent, I could usually bet on these characteristics:

  • 部署到AWS(毕竟我们是AWS咨询公司…)
  • 结合使用无服务器框架和Node.js或Python
  • 可能将生产部署到多个地区
  • 可以将不同阶段部署到不同帐户

另外,我意识到我还需要以下其他选择:

  • 可能需要从实际工作中“禁用”开发/暂存
  • 每个分支可能需要一个开发环境

最后,我们决定了以下部署策略:

  • 生产通过标签部署 master
  • 登台部署在提交/合并到 master
  • 开发人员部署应可用于所有其他分支机构(在本文中我们将不实施此分支机构)

我的根基是作为软件开发人员,因此使事情可重用是此时的一项核心技能。一个好的模板将使其在预期的情况下超级容易,并且相当适合于其他用途。目标是:

  • 每个阶段一个脚本。这意味着只有一个测试脚本,一个构建脚本和一个部署脚本。哦,保持干燥。
  • 作业应尽可能相似,并且应通过顶级变量来调整差异。

Let’s focus on that single script per stage. We’re not going to cover how to write the deployment script, but we’ll focus on the deploy stage. But let’s say we start with a deployment job like this:

<pre><code>deploy:生产:阶段:部署脚本:| #假设在AWS中$ {DEPLOYMENT_ROLE}#安装依赖项#使用$ {STAGE_NAME} $ {REGION}变量运行无服务器部署:DEPLOYMENT_ROLE:arn:aws:iam :: 1234567890:role / gitlab-ci-deployment STAGE_NAME:生产区域:我们-east-1帐户:仅“ 1234567890”:-标签</code></pre>

现在,我们可以复制和调整此文件以进行登台和开发,但这不是我们要追求的。首先,让我们将脚本分解为可重用的块,并将其用于登台部署中:

<pre><code>.deployment_script: <b>&deployment_script</b>阶段:部署脚本: #假定在AWS中$ {DEPLOYMENT_ROLE}#安装依赖项#使用$ {STAGE_NAME} $ {REGION} deploy:production运行无服务器部署:<<: <b>*deployment_script</b>变量:DEPLOYMENT_ROLE:arn:aws:iam :: 1234567890:role / gitlab-ci-deployment STAGE_NAME:产品区域:us-east-1帐户:“ 1234567890”产量:“ true”仅:-标签部署:阶段:<<: <b>*deployment_script</b>变量:DEPLOYMENT_ROLE:arn:aws:iam :: 0987654321:role / gitlab-ci-deployment STAGE_NAME:暂存区域:us-east-1帐户:仅“ 0987654321”:-主</code></pre>

Using YAML anchors and references, we can inject the script into 所有 of our deployment jobs. Notice that the deployment script is disabled. This is because we don’t want it to run in parallel with 所有 of our intended jobs. We also added a PRODUCTION environment variable to just the production deploy to 所有ow our script to pick that up too. If your 码 knows about this, you can use this to turn on/off production only features. Now, we can make this cleaner and easier for our developers by pulling 所有 of the variables to a top level variables map at the top of the file:

<pre><code>变量:PROD_ACCOUNT:“ 1234567890” PROD_STAGE_NAME:产品PROD_REGION:us-east-1 STAGING_ACCOUNT:“ 0987654321” STAGING_STAGE_NAME:登台STAGING_REGION:us-east-1 。:<b>&deployment_script</b>阶段:部署脚本: #假设在AWS中$ {DEPLOYMENT_ROLE}#安装依赖项#使用$ {STAGE_NAME},$ {REGION}和$ {ACCOUNT} deploy:production运行无服务器部署:<<: <b>*deployment_script</b>变量:DEPLOYMENT_ROLE:“ arn:aws:iam :: $$ {PROD_ACCOUNT}:role / gitlab-ci-deployment” STAGE_NAME:$ {PROD_STAGE_NAME} REGION:$ {PROD_REGION}帐户:$ {PROD_ACCOUNT}生产:仅“ true”: -标签部署:阶段:<<: <b>*deployment_script</b>变量:DEPLOYMENT_ROLE:“ arn:aws:iam :: $$ {STAGING_ACCOUNT}:role / gitlab-ci-deployment” STAGE_NAME:$ {STAGING_STAGE_NAME} REGION:$ {STAGING_REGION}帐户:$ {STAGING_ACCOUNT}仅:-主用户</code></pre>

现在,它看起来更具可重用性,并且我们已经实现了我们的第二个目标,即使工作非常相似并由顶级变量控制。这样,适合模板用例的任何人都可以轻松重用它。我们可以轻松添加开发环境,但是为了说明多区域生产部署,我们将跳过该环境:

<pre><code>变量:PROD_ACCOUNT:“ 1234567890” PROD_STAGE_NAME:产品PROD1_REGION:us-east-1 PROD2_REGION:us-west-2 STAGING_ACCOUNT:“ 0987654321” STAGING_STAGE_NAME:登台STAGING_REGION:us-east-1 。:<b>&deployment_script</b>阶段:部署脚本: #假定在AWS中$ {DEPLOYMENT_ROLE}#安装依赖项#使用$ {STAGE_NAME},$ {REGION}和$ {ACCOUNT}运行无服务器部署。production_variablesDEPLOYMENT_ROLE:“ arn:aws:iam :: $$ {PROD_ACCOUNT}:role / gitlab-ci-deployment“ STAGE_NAME:$ {PROD_STAGE_NAME}帐户:$ {PROD_ACCOUNT} PRODUCTION:” true“ deploy:production_1:<b>&deploy_production</b> <<: <b>*deployment_script</b> variables: <<: <b>* production_variables</b>区域:仅$ {PROD1_REGION}:-标签部署:生产_2:<<: <b>*deploy_production</b> variables: <<: <b>* production_variables</b>区域:$ {PROD2_REGION}部署:阶段:<<: <b>*deployment_script</b>变量:DEPLOYMENT_ROLE:“ arn:aws:iam :: $$ {STAGING_ACCOUNT}:role / gitlab-ci-deployment” STAGE_NAME:$ {STAGING_STAGE_NAME} REGION:$ {STAGING_REGION}帐户:$ {STAGING_ACCOUNT}仅:-主用户</code></pre>

Notice that we have changed the job names to reflect having multiple regions. In addition, we are making use of YAML anchors and references to copy the entire deploy:production_1 job into deploy:production_2 and then we just override the REGION variable. This makes adding additional regions super easy.

此时更有用的是,只要您使脚本足够灵活,就可以将其作为模板分发给开发团队。如果他们的项目完全适合脚本和配置,则只需要为顶层变量填写正确的值即可。对于那些需要不同的东西的人,他们应该应该能够调整脚本。现在,我们只需要解决确保他们实际使用模板的问题...

使用GitLab和AWS Lambda进行自动江苏体彩注入

I was inspired by GitHub’s option to select a .gitignore and license during the repo creation process. What if we could have that for 江苏体彩? Forking GitLab and figuring out how to hack this in did not sound like a quick or easy thing to do. However, after a little research, I found that we could use a system hook to trigger a Lambda that could inject the desired template via the commit API. This part is not as interesting to read about, so we did one better: we have open sourced this tool so you can deploy it in your environment. Check out the repo 这里。而且,如果您正在寻找可以帮助您实现这些以及其他出色的自动化和AWS解决方案的人员,我们很乐意与您联系。欢迎随时与我们联系 [email protected] 更多。谢谢阅读!

作者
罗布·里贝罗(Rob Ribeiro)Trek10
罗伯·里贝罗

团队支持首席架构师