Jenkins export and import jobs 迁移导出导入任务实践小结
我遇到的Jenkins迁移项目并非可以通过简单的物理文件复制就可以轻松解决，需要考虑上千个不同项目的jobs分离，Jenkins 1.x和2.x大版本兼容性，Jenkins Plugins插件，Jenkins Credentials凭证，Jenkins Restrict节点约束，按view分类不同项目的jobs等各种因素。这次对Jenkins迁移做了大量的研究和实践，希望总结出来的方法能对各位有帮助。
Jenkins export and import jobs 迁移导出导入任务实践小结
2020年01月06日 - 初稿
阅读原文 - https://wsgzao.github.io/post/jenkins-import/
export and import jobs in Jenkins
Is it possible to exchange jobs between 2 different Jenkins’? I’m searching for a way to export/import jobs.
Jenkins has a rather good wiki, albeit hard to read when you’re new to CI software…
They offer a simple solution for moving jobs between servers
The trick probably was the need to reload config from the Jenkins Configuration Page.
Probably use jenkins command line is another option, see https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI
- create-job: Creates a new job by reading stdin as a configuration XML file.
- get-job: Dumps the job definition XML to stdout
So you can do
java -jar jenkins-cli.jar -s http://server get-job myjob > myjob.xml
It works fine for me and I am used to store in inside my version control system
$ curl -s http://OLD_JENKINS/job/JOBNAME/config.xml | curl -X POST 'http://NEW_JENKINS/createItem?name=JOBNAME' --header "Content-Type: application/xml" -d @-
$ curl -s http:///<USER>:<API_TOKEN>@OLD_JENKINS/job/JOBNAME/config.xml | curl -X POST 'http:///<USER>:<API_TOKEN>@NEW_JENKINS/createItem?name=JOBNAME' --header "Content-Type: application/xml" -d @-
With Crumb, if CSRF is active (see details here):
Get crumb with:
$ CRUMB_OLD=$(curl -s 'http://<USER>:<API_TOKEN>@OLD_JENKINS/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')
Apply crumb with
$ curl -s -H $CRUMB_OLD http:///<USER>:<API_TOKEN>@OLD_JENKINS/job/JOBNAME/config.xml | curl -X POST -H $CRUMB_NEW 'http:///<USER>:<API_TOKEN>@NEW_JENKINS/createItem?name=JOBNAME' --header "Content-Type: application/xml" -d @-
There’s a plugin called Job Import Plugin that may be what you are looking for. I have used it. It does have issues with importing projects from a server that doesn’t allow anonymous access.
If you have command line access to both, you can do the procedure already mentioned by Khez for Moving, Copying and Renaming Jenkins Jobs.
Go to your Jenkins server’s front page, click on REST API at the bottom of the page:
To create a new job, post
config.xml to this URL with query parameter
name=JOBNAME. You need to send a
Content-Type: application/xml header. You’ll get
200 status code if the creation is successful, or
4xx/5xx code if it fails.
config.xml is the format Jenkins uses to store the project in the file system, so you can see examples of them in the Jenkins home directory, or by retrieving the XML configuration of existing jobs from
In my Jenkins instance (version 1.548) the configuration file is at:
Owned by jenkins user and jenkins group with 644 permissions. Copying the file to and from here should work. I haven’t tried changing it directly but have backed-up the config from this spot in case the project needs to be setup again.
Job Import plugin is the easy way here to import jobs from another Jenkins instance. Just need to provide the URL of the source Jenkins instance. The Remote Jenkins URL can take any of the following types of URLs:
http://$JENKINS– get all jobs on remote instance
http://$JENKINS/job/$JOBNAME– get a single job
http://$JENKINS/view/$VIEWNAME– get all jobs in a particular view
Thanks to Larry Cai’s answer I managed to create a script to backup all my Jenkins jobs. I created a job that runs this every week. In case someone finds it useful, here it is:
Jenkins export jobs to a directory
for f in *.xml;
Simple php script worked for me.
// add all job codes in the array
This does not work for existing jobs, however there is Jenkins job builder.
This allows one to keep job definitions in yaml files and in a git repo which is very portable.
For those of us in the Windows world who may or may not have Bash available, here’s my PowerShell port of Katu and Larry Cai‘s approach. Hope it helps someone.
##### Config vars #####
It is very easy just download plugin name
Enter the URL of your Remote Jenkins server and it will import the jobs automatically
The most easy way, with direct access to the machine is to copy the job folder from first jenkins to another one (you can exclude workspaces –
workspace folder), because the whole job configuration is stored in the xml file on the disk.
Then in the new jenkins just
reload configuration in the global settings (admin access is required) should be enough, if not, then you will need to restart Jenkins tool.
Another way can be to use plugins mentioned above this post.
– in case you can probably also exclude
- 官方的Moving/copying/renaming jobs，即所谓的物理文件迁移，如果情况复杂此方案不推荐
- 基于Jenkins CLI，该方案需要依赖jenkins-cli.jar包括java使用范围具有一定局限性，不推荐
- 基于类似Job Import Plugin插件形式，该类方案普适性较差，不推荐
- 基于Jenkins REST API，核心是获取
Many objects of Jenkins provide the remote access API. They are available at /…/api/ where “…” portion is the object for which you’d like to access.
Access data exposed in HTML as XML for machine consumption. Schema is also available.
You can also specify optional XPath to control the fragment you’d like to obtain (but see below). For example, ../api/xml?xpath=//.
For XPath that matches multiple nodes, you need to also specify the “wrapper” query parameter to specify the name of the root XML element to be create so that the resulting XML becomes well-formed.
Similarly exclude query parameter can be used to exclude nodes that match the given XPath from the result. This is useful for trimming down the amount of data you fetch (but again see below). This query parameter can be specified multiple times.
XPath filtering is powerful, and you can have it only return a very small data, but note that the server still has to build a full DOM of the raw data, which could cause a large memory spike. To avoid overloading the server, consider using the tree parameter, or use the xpath parameter in conjunction with the tree parameter. When used together, the result of the tree parameter filtering is built into DOM, then the XPath is applied to compute the final return value. In this way, you can often substantially reduce the size of DOM built in memory.
Access the same data as Python for Python clients. This can be parsed into Python object as eval(urllib.urlopen(“…”).read()) and the resulting object tree is identical to that of JSON. However, when you do this, beware of the security implication. If you are connecting to a non-trusted Jenkins, the server can send you malicious Python programs.
In Python 2.6 or later you can safely parse this output using ast.literal_eval(urllib.urlopen(“…”).read())
For more information about remote API in Jenkins, see the documentation.
使用curl和Jenkins REST API
使用Jenkins REST API建议关闭CSRF防护
# To retrieve the job config.xml
- username API_TOKEN with your username and password/API_Token
- jenkinshost with your Jenkins URL
- jobname with the name of the job that you created via the UI
- 支持按view或者job export导出job config.xml配置信息
- 支持按view或者job import导入job config.xml配置信息
archive jenkins setting and plugins
Jenkins API Token
New API Token system should allow tokens to be created for service accounts
How to create a job using the REST API and cURL?