Buildbot in examples
I needed to set up the process of assembling and delivering software packages from the Git repository to the site. And having seen, not so long ago, here on Habr the article on buildbot (the link at the end) decided to try it and apply for this.
Since buildbot is a distributed system, it will be logical for each architecture and OS to make a separate assembly host. In our case, these will be LXC containers (in the case of linux) and qemu (in the case of windows):
- vm-srv-build1 - centos 7, there will be a buildbot master and one of the workers
- vm-srv-build2 - debian 10, for building DEB packages
- vm-srv-build3 - windows 10, for assembly, you yourself understand what
We will collect the Rac GUI - a graphical face to 1C rac for managing a cluster of servers. Under Linux, standard tools for each OS will be used, freewrap is used to build an exe file for windows from a tcl script.
Installation
GNU / Linux
For installation, documentation on the network is enough 1 , 2 . And it does not cause any special problems:
For the master:
pip3 install buildbot pip3 install twisted pip3 install autobahn pip3 install pysqlite3 pip3 install sqlalchemy sqlalchemy-migrate pip3 install buildbot-www buildbot-grid-view buildbot-console-view buildbot-waterfall-view pip3 install python-dateutil
For the "workers", this is sufficient:
pip3 install buildbot-worker
Of course, it would be more correct to collect packages for each OS, but this is not included in the scope of the article. We also omit the description of configuring containers for work, I only note that I use ProxMox VE. And you also need to install packages for each axis required for assembly (centos: rpmdevtools, etc.; debian: build-essential, dh-make, pbuilder, etc.)
Project assembly and buildbot services will be launched on behalf of an unprivileged user, so you need to create it on all the hosts involved in the process:
adduser buildbot
Next, configure the automatic launch of services, respectively, on each of the hosts (containers):
Systemd unit to run the wizard:
touch /etc/systemd/buildbot-master.service [Unit] Description=BuildBot master service After=network.target [Service] User=buildbot Group=buildbot WorkingDirectory=/home/buildbot/master ExecStart=/usr/local/bin/buildbot start --nodaemon ExecReload=/bin/kill -HUP $MAINPID [Install] WantedBy=multi-user.target
and "worker"
touch /etc/systemd/buildbot-worker.service [Unit] Description=BuildBot worker service After=network.target [Service]master User=buildbot Group=buildbot WorkingDirectory=/home/buildbot/worker ExecStart=/usr/local/bin/buildbot-worker start --nodaemon [Install] WantedBy=buildbot-master.service
Since all scripts (in our case) are in / usr / local /, you should write the path to them in environment variables:
nano /root/.bash_profile PATH=$PATH:$HOME/.local/bin:$HOME/bin:/usr/local/bin
After that, you can create a directory infrastructure for the "workers" (on all hosts), for this, register under the buildbot user and execute the following commands:
On the first host vm-srv-build1:
su - buildbot mkdir /home/buildbot/worker cd ~ buildbot-worker create-worker --umask=0o22 --keepalive=60 worker vm-srv-build1:4000 CentOS 123456
On the second host, vm-srv-build2:
su - buildbot mkdir /home/buildbot/worker cd ~ buildbot-worker create-worker --umask=0o22 --keepalive=60 worker vm-srv-build1:4000 Debian-10 123456
On production hosts, the buildbot-worker service can be started
systemctl start buildbot-worker
MS Windows
As a "working" build for windows, a virtual machine with the latest Win10 release will be used.
To work, you need:
- Python
- Pywin
- Git
- To install all sorts of different things in python, I needed to download Build Tools for Visual Studio 2019
- 7zip
After all of the above is installed, you can install buildbot itself:
pip install buildbot-worker
Create a working directory
md c:\worker
And run
buildbot-worker start c:\worker
If everything works (see the c: \ worker \ twistd.log log), then you can register our "worker" as a service by adding an item with the working directory to the registry (commands are executed in powershell running as administrator):
buildbot_worker_windows_service.exe --user VM-SRV-BUILD3\buildbot --password 123456 --startup auto install New-ItemProperty -path Registry::HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\BuildBot\Parameters -Name directories -PropertyType String -Value c:\worker
And you can run a servant
Start-Service buildbot
That’s all with the “workers”, then you can leave them alone, all control goes from the master.
Setup Wizard
To begin with, we will create the infrastructure for the wizard (on the main host), for this we register under the buildbot user and execute the following commands:
su - buildbot mkdir /home/buildbot/master cd ~ buildbot create-master master
For finished packages, create the builds directory
mkdir /home/buildbot/builds
The master.cfg file was created in the / home / buildbot / master / directory. This file is a python code and contains a description of all the mechanisms of the system, we will continue to work with it.
nano /home/buildbot/master/master.cfg
import os, re from buildbot.plugins import steps, util, schedulers, worker, changes, reporters c= BuildmasterConfig ={} # . c['workers'] = [ worker.Worker('CentOS', '123456'), worker.Worker('Debian-10', '123456'), worker.Worker('Windows-10', '123456')] c['protocols'] = {'pb': {'port': 4000}} # c['change_source'] = [] c['change_source'].append(changes.GitPoller( repourl = 'https://bitbucket.org/svk28/rac-gui.git', project = 'Rac-GUI', branches = True, pollInterval = 60 ) ) # c['schedulers'] = [] c['schedulers'].append(schedulers.SingleBranchScheduler( name="Rac-GUI-schedulers", change_filter=util.ChangeFilter(branch='master'), builderNames=["Rac-GUI-RPM-builder", "Rac-GUI-DEB-builder", "Rac-GUI-WIN-builder"], properties = {'owner': 'admin'} ) ) @util.renderer ######################################3 # RPM- rac_gui_build_RPM = util.BuildFactory() rac_gui_build_RPM.addStep(steps.Git( repourl = 'https://bitbucket.org/svk28/rac-gui.git', workdir = 'rac-gui', haltOnFailure = True, submodules = True, mode='full', progress = True) )
To automate the assembly of packages of different versions, so as not to have to climb into the code of the master.cfg file, in the main script of the rac_gui.tcl program in the header, lines with the current version and release were added:
###################################################### # Rac GUI ... # version: 1.0.3 # release: 1
And based on these lines, buildbot will number the packages. To pull data, use the console grep call. In buildbot, you simply cannot define variables for “workers” (in any case, I have not found how). To do this, use properties. Those. in the assembly process, add the steps to determine the version and release, and accordingly, set the version and release properties. Properties can be set in various ways, in this case a call to the console command:
# rac_gui_build_RPM.addStep( steps.SetPropertyFromCommand( command="grep version ../rac-gui/rac_gui.tcl | grep -oE '\\b[0-9]{1,2}\\.[0-9]{1,2}\\.[0-9]{1,2}\\b'", property="version" ) ) # rac_gui_build_RPM.addStep( steps.SetPropertyFromCommand( command="grep release ../rac-gui/rac_gui.tcl | grep -oE '\\b[0-9]{1,3}\\b'", property="release" ) )
Substitute the obtained values by calling util.Interpolate ().
# rac_gui_build_RPM.addStep( steps.ShellCommand( command=["tar", "czf", util.Interpolate("/home/buildbot/rpmbuild/SOURCES/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"), "../rac-gui"] ) )
It should be noted that, since the host is also used for manual assembly of packages, the assembly will take place along standard paths.
# spec rac_gui_build_RPM.addStep(steps.ShellCommand( command=["cp", "../rac-gui/rac_gui.spec", "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"]))
To set the correct release and version numbers, use the standard sed call, i.e. the command replaces the values inside the spec file with the necessary
# rac_gui_build_RPM.addStep(steps.ShellCommand( command=["sed", "-i", util.Interpolate("s/.*Version:.*/Version:\t%(prop:version)s/"), "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"])) # rac_gui_build_RPM.addStep(steps.ShellCommand( command=["sed", "-i", util.Interpolate("s/.*Release:.*/Release:\t%(prop:release)s/"), "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"])) # rac_gui_build_RPM.addStep(steps.RpmBuild( specfile="/home/buildbot/rpmbuild/SPECS/rac_gui.spec", dist='.el5', topdir='/home/buildbot/rpmbuild', builddir='/home/buildbot/rpmbuild/build', rpmdir='/home/buildbot/rpmbuild/RPMS', sourcedir='/home/buildbot/rpmbuild/SOURCES' ) )
The finished assembled package and archive with source codes are copied to the master. But you can immediately copy files from your desktop to your repository or website.
# rac_gui_build_RPM.addStep( steps.FileUpload( workersrc=util.Interpolate("/home/buildbot/rpmbuild/RPMS/noarch/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm"), masterdest=util.Interpolate("/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm") ) ) rac_gui_build_RPM.addStep( steps.FileUpload( workersrc=util.Interpolate("/home/buildbot/rpmbuild/SOURCES/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"), masterdest=util.Interpolate("/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz") ) )
Let's start the process of copying collected packages to FTP hosting on the wizard. To do this, use the tcl script .
rac_gui_build_RPM.addStep( steps.MasterShellCommand( command=["/usr/local/bin/deploy-ftp.tcl", util.Interpolate("--local-file=/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm"), util.Interpolate("--remote-file=uploads/rac-gui/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm")] ) ) rac_gui_build_RPM.addStep( steps.MasterShellCommand( command=["/usr/local/bin/deploy-ftp.tcl", util.Interpolate("--local-file=/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"), util.Interpolate("--remote-file=uploads/rac-gui/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz")] ) )
On it with RPM finished. Now let's start the description of the assembly algorithm for the DEB package. Since the processes of building packages for different systems are independent of each other, many steps will be repeated.
rac_gui_build_DEB = util.BuildFactory() rac_gui_build_DEB.addStep(steps.Git( repourl = 'https://bitbucket.org/svk28/rac-gui.git', haltOnFailure = True, submodules = True, mode='full', workdir='build', progress = True) ) # rac_gui_build_DEB.addStep( steps.SetPropertyFromCommand( command="grep version rac_gui.tcl | grep -oE '\\b[0-9]{1,2}\\.[0-9]{1,2}\\.[0-9]{1,2}\\b'", property="version" ) ) # rac_gui_build_DEB.addStep( steps.SetPropertyFromCommand( command="grep release rac_gui.tcl | grep -oE '\\b[0-9]{1,3}\\b'", property="release" ) ) # rac_gui_build_DEB.addStep(steps.ShellCommand( command=["mv", "rac_gui.tcl", "racgui"]))
For the RPM package, part of the following procedures is done by rpm itself during assembly and is described inside the spec, for debian you have to do it here:
# rac_gui_build_DEB.addStep(steps.ShellCommand( command=["sed", "-i", "s+^set\ dir(lib)+set\ dir(lib)\ /usr/share/rac-gui/lib ;#+g", "racgui"])) # rac_gui_build_DEB.addStep(steps.ShellCommand( command=["sed", "-i", "s+\[pwd\]+/usr/share/rac-gui+g", "racgui"])) # rac_gui_build_DEB.addStep(steps.ShellCommand( command=["tar", "czf", util.Interpolate("../rac-gui_%(prop:version)s.orig.tar.gz"), "."])) # rac_gui_build_DEB.addStep(steps.ShellCommand( command=["dpkg-buildpackage"])) # rac_gui_build_DEB.addStep( steps.FileUpload( workersrc=util.Interpolate("../rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb"), masterdest=util.Interpolate("/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb") ) ) rac_gui_build_DEB.addStep( steps.MasterShellCommand( command=["/usr/local/bin/deploy-ftp.tcl", util.Interpolate("--local-file=/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb"), util.Interpolate("--remote-file=uploads/rac-gui/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb")] ) )
And DEB is done, now windows!
rac_gui_build_WIN = util.BuildFactory() rac_gui_build_WIN.addStep(steps.Git( repourl = 'https://bitbucket.org/svk28/rac-gui.git', haltOnFailure = True, submodules = True, mode='full', workdir='build', progress = True) )
Since windows does not have grep and sed (or is there?), We will use powershell
# rac_gui_build_WIN.addStep( steps.SetPropertyFromCommand( command="powershell -command \"((Get-Content .\\rac_gui.tcl | Select-String -Pattern 'version:') -split '\\s')[2]", property="version" ) ) # rac_gui_build_WIN.addStep( steps.SetPropertyFromCommand( command="powershell -command \"((Get-Content .\\rac_gui.tcl | Select-String -Pattern 'release:') -split '\\s')[2]", property="release" ) ) # rac_gui_build_WIN.addStep(steps.ShellCommand( command=["c:\\bin\\freewrap.exe", "rac_gui.tcl"])) # , rac_gui_build_WIN.addStep(steps.ShellCommand( command=["c:\\Program Files\\7-zip\\7z.exe", "a", "-r", util.Interpolate("..\\rac-gui_%(prop:version)s-%(prop:release)s.win.zip"), "..\\build"])) # rac_gui_build_WIN.addStep( steps.FileUpload( workersrc=util.Interpolate("..\\rac-gui_%(prop:version)s-%(prop:release)s.win.zip"), masterdest=util.Interpolate("/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s.win.zip") ) ) # rac_gui_build_WIN.addStep( steps.MasterShellCommand( command=["/usr/local/bin/deploy-ftp.tcl", util.Interpolate("--local-file=/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s.win.zip"), util.Interpolate("--remote-file=uploads/rac-gui/rac-gui_%(prop:version)s-%(prop:release)s.win.zip")] ) ) # c['builders'] = [ util.BuilderConfig(name="Rac-GUI-RPM-builder", workername='CentOS', factory=rac_gui_build_RPM), util.BuilderConfig(name="Rac-GUI-DEB-builder", workername='Debian-10', factory=rac_gui_build_DEB), util.BuilderConfig(name="Rac-GUI-WIN-builder", workername='Windows-10', factory=rac_gui_build_WIN), ]
To notify about the state of the assembly process, we will use e-mail
c['services'] = [] template=u'''\ <h4>Build status: {{ summary }}</h4> <p> Worker used: {{ workername }}</p> {% for step in build['steps'] %} <p> {{ step['name'] }}: {{ step['result'] }}</p> {% endfor %} <p><b> -- The Buildbot</b></p> ''' mailNotifier = reporters.MailNotifier(fromaddr="builder@domain.ru", sendToInterestedUsers=False, mode=('all'), extraRecipients=["admin@domain.ru"], relayhost="mail.domain.ru", smtpPort=587, smtpUser="builder@domain.ru", smtpPassword="******", messageFormatter=reporters.MessageFormatter( template=template, template_type='html', wantProperties=True, wantSteps=True)) c['services'].append(mailNotifier) # c['title'] = "The process of bulding" c['titleURL'] = "http://vm-srv-build1:80/" c['buildbotURL'] = "http://vm-srv-build1/" c['www'] = dict(port=80, plugins=dict(waterfall_view={}, console_view={}, grid_view={})) c['db'] = { 'db_url' : "sqlite:///state.sqlite" }
We save the file and you can try to start the wizard service:
systemctl restart buildbot-master
In the log, we check that everything is in order with the config and everything works as usual. All our employees should now connect, as will be happily reported in the '' '' '' /home/buildbot/master/twistd.log '' '' 'log:
2019-07-24 16:50:35+0300 [-] Loading buildbot.tac... 2019-07-24 16:50:35+0300 [-] Loaded. 2019-07-24 16:50:35+0300 [-] twistd 19.2.1 (/usr/bin/python3.6 3.6.8) starting up. 2019-07-24 16:50:35+0300 [-] reactor class: twisted.internet.epollreactor.EPollReactor. 2019-07-24 16:50:35+0300 [-] Starting BuildMaster -- buildbot.version: 2.3.1 2019-07-24 16:50:35+0300 [-] Loading configuration from '/home/buildbot/master/master.cfg' 2019-07-24 16:50:36+0300 [-] /usr/local/lib/python3.6/site-packages/buildbot/config.py:90: buildbot.config.ConfigWarning: [0.9.0 and later] `buildbotNetUsageData` is not configured and defaults to basic. This parameter helps the buildbot development team to understand the installation base. No personal information is collected. Only installation software version info and plugin usage is sent. You can `opt-out` by setting this variable to None. Or `opt-in` for more information by setting it to "full". 2019-07-24 16:50:36+0300 [-] Setting up database with URL 'sqlite:/state.sqlite' 2019-07-24 16:50:36+0300 [-] setting database journal mode to 'wal' 2019-07-24 16:50:36+0300 [-] adding 1 new services, removing 0 2019-07-24 16:50:36+0300 [-] adding 1 new change_sources, removing 0 2019-07-24 16:50:36+0300 [-] gitpoller: using workdir '/home/buildbot/master/gitpoller-work' 2019-07-24 16:50:36+0300 [-] adding 3 new builders, removing 0 2019-07-24 16:50:36+0300 [-] adding 1 new schedulers, removing 0 2019-07-24 16:50:36+0300 [-] initializing www plugin 'waterfall_view' 2019-07-24 16:50:36+0300 [-] initializing www plugin 'console_view' 2019-07-24 16:50:36+0300 [-] initializing www plugin 'grid_view' 2019-07-24 16:50:36+0300 [-] NOTE: www plugin 'sitenav' is installed but not configured 2019-07-24 16:50:36+0300 [-] initializing www plugin 'waterfall_view' 2019-07-24 16:50:36+0300 [-] initializing www plugin 'console_view' 2019-07-24 16:50:36+0300 [-] initializing www plugin 'grid_view' 2019-07-24 16:50:36+0300 [-] NOTE: www plugin 'sitenav' is installed but not configured 2019-07-24 16:50:36+0300 [-] BuildbotSite starting on 80 2019-07-24 16:50:36+0300 [-] Starting factory <buildbot.www.service.BuildbotSite object at 0x7fe31c2657b8> 2019-07-24 16:50:36+0300 [-] adding 3 new workers, removing 0 2019-07-24 16:50:36+0300 [-] PBServerFactory starting on 4000 2019-07-24 16:50:36+0300 [-] Starting factory <twisted.spread.pb.PBServerFactory object at 0x7fe31c147470> 2019-07-24 16:50:37+0300 [-] BuildMaster is running 2019-07-24 16:50:37+0300 [-] buildbotNetUsageData: sending {'installid': 'b6193b126b96689351d2fe95787c5a03fc0879f9', 'versions': {'Python': '3.6.8', 'Buildbot': '2.3.1', 'Twisted': '19.2.1'}, 'platform': {'platform': 'Linux-4.15.18-10- pve-x86_64-with-centos-7.6.1810-Core', 'system': 'Linux', 'machine': 'x86_64', 'processor': 'x86_64', 'python_implementation': 'CPython', 'version': '#1 SMP PVE 4.15.18-32', 'distro': 'centos:7'}, 'plugins': {'buildbot/worker/base/Worker': 3, 'buildbot/config/BuilderConfig': 3, 'buildbot/schedulers/basic/SingleBranchScheduler': 1, 'buildbot/reporters/mail/MailNotifier': 1, 'buildbot/changes/gitpoller/GitPoller': 1, 'buildbot/steps/worker/MakeDirectory': 1, 'buildbot/steps/source/git/Git': 3, 'buildbot/steps/shell/ShellCommand': 9, 'buildbot/steps/package/rpm/rpmbuild/RpmBuild': 1}, 'db': 'sqlite', 'mq': 'simple', 'www_plugins': ['waterfall_view', 'console_view', 'grid_view']} 2019-07-24 16:50:37+0300 [Broker,0,127.0.0.1] worker 'CentOS' attaching from IPv4Address(type='TCP', host='127.0.0.1', port=37332) 2019-07-24 16:50:37+0300 [Broker,0,127.0.0.1] Got workerinfo from 'CentOS' 2019-07-24 16:50:37+0300 [-] bot attached 2019-07-24 16:50:37+0300 [Broker,0,127.0.0.1] Worker CentOS attached to Rac-GUI-RPM-builder 2019-07-24 16:50:37+0300 [-] buildbotNetUsageData: buildbot.net said: ok 2019-07-24 16:50:39+0300 [Broker,1,192.168.55.15] worker 'Windows-10' attaching from IPv4Address(type='TCP', host='192.168.5.145', port=49831) 2019-07-24 16:50:39+0300 [Broker,1,192.168.55.15] Got workerinfo from 'Windows-10' 2019-07-24 16:50:40+0300 [-] bot attached 2019-07-24 16:50:40+0300 [Broker,1,192.168.55.15] Worker Windows-10 attached to Rac-GUI-WIN-builder 2019-07-24 16:50:41+0300 [Broker,2,192.168.55.99] worker 'Debian-10' attaching from IPv4Address(type='TCP', host='192.168.5.9', port=44430) 2019-07-24 16:50:41+0300 [Broker,2,192.168.55.99] Got workerinfo from 'Debian-10' 2019-07-24 16:50:41+0300 [-] bot attached 2019-07-24 16:50:41+0300 [Broker,2,192.168.55.99] Worker Debian-10 attached to Rac-GUI-DEB-builder
This completes the setup process. You can view the current status through the web-face. Where you can also see build errors, kick a frozen process if something went wrong, etc.
Right after the launch of our hard workers, you can see through the menu "Builds" -> "Workers"
After the first build process is completed (i.e. changes to the Git repository), the state of the processes will appear on the first page.
If you poke the mouse in the desired line, a page opens with the current state of the process, where you can see what is happening, what errors, etc.
import os, re from buildbot.plugins import steps, util, schedulers, worker, changes, reporters c= BuildmasterConfig ={} c['workers'] = [ worker.Worker('CentOS', '123456'), worker.Worker('Debian-10', '123456'), worker.Worker('Windows-10', '123456')] c['protocols'] = {'pb': {'port': 4000}} c['change_source'] = [] c['change_source'].append(changes.GitPoller( repourl = 'https://bitbucket.org/svk28/rac-gui.git', project = 'Rac-GUI', branches = True, pollInterval = 600 )) # c['schedulers'] = [] c['schedulers'].append(schedulers.SingleBranchScheduler( name="Rac-GUI-schedulers", change_filter=util.ChangeFilter(branch='master'), builderNames=["Rac-GUI-RPM-builder", "Rac-GUI-DEB-builder", "Rac-GUI-WIN-builder"], properties = {'owner': 'admin'} )) @util.renderer def get_name_version_release(props): prog_name = "rac-gui" prog_version = "1.0.3" prog_release = "3" return { "prog_name": prog_name #"prog_version": prog_version, #"prog_release": prog_release } rac_gui_build_RPM = util.BuildFactory() rac_gui_build_RPM.addStep(steps.Git( repourl = 'https://bitbucket.org/svk28/rac-gui.git', workdir = 'rac-gui', haltOnFailure = True, submodules = True, mode='full', progress = True) ) # rac_gui_build_RPM.addStep( steps.SetPropertyFromCommand( command="grep version ../rac-gui/rac_gui.tcl | grep -oE '\\b[0-9]{1,2}\\.[0-9]{1,2}\\.[0-9]{1,2}\\b'", property="version" ) ) # rac_gui_build_RPM.addStep( steps.SetPropertyFromCommand( command="grep release ../rac-gui/rac_gui.tcl | grep -oE '\\b[0-9]{1,3}\\b'", property="release" ) ) rac_gui_build_RPM.addStep(steps.ShellCommand( command=["tar", "czf", util.Interpolate("/home/buildbot/rpmbuild/SOURCES/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"), "../rac-gui"])) rac_gui_build_RPM.addStep(steps.ShellCommand( command=["cp", "../rac-gui/rac_gui.spec", "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"])) rac_gui_build_RPM.addStep(steps.ShellCommand( command=["sed", "-i", util.Interpolate("s/.*Version:.*/Version:\t%(prop:version)s/"), "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"])) rac_gui_build_RPM.addStep(steps.ShellCommand( command=["sed", "-i", util.Interpolate("s/.*Release:.*/Release:\t%(prop:release)s/"), "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"])) rac_gui_build_RPM.addStep(steps.RpmBuild( specfile="/home/buildbot/rpmbuild/SPECS/rac_gui.spec", dist='.el5', topdir='/home/buildbot/rpmbuild', builddir='/home/buildbot/rpmbuild/build', rpmdir='/home/buildbot/rpmbuild/RPMS', sourcedir='/home/buildbot/rpmbuild/SOURCES' )) # rac_gui_build_RPM.addStep( steps.FileUpload( workersrc=util.Interpolate("/home/buildbot/rpmbuild/RPMS/noarch/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm"), masterdest=util.Interpolate("/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm") ) ) rac_gui_build_RPM.addStep( steps.FileUpload( workersrc=util.Interpolate("/home/buildbot/rpmbuild/SOURCES/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"), masterdest=util.Interpolate("/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz") ) ) rac_gui_build_RPM.addStep( steps.MasterShellCommand( command=["/usr/local/bin/deploy-ftp.tcl", util.Interpolate("--local-file=/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm"), util.Interpolate("--remote-file=uploads/rac-gui/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm")] ) ) rac_gui_build_RPM.addStep( steps.MasterShellCommand( command=["/usr/local/bin/deploy-ftp.tcl", util.Interpolate("--local-file=/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"), util.Interpolate("--remote-file=uploads/rac-gui/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz")] ) ) #################################### ## DEB #################################### rac_gui_build_DEB = util.BuildFactory() rac_gui_build_DEB.addStep(steps.Git( repourl = 'https://bitbucket.org/svk28/rac-gui.git', haltOnFailure = True, submodules = True, mode='full', workdir='build', progress = True) ) # rac_gui_build_DEB.addStep( steps.SetPropertyFromCommand( command="grep version rac_gui.tcl | grep -oE '\\b[0-9]{1,2}\\.[0-9]{1,2}\\.[0-9]{1,2}\\b'", property="version" ) ) # rac_gui_build_DEB.addStep( steps.SetPropertyFromCommand( command="grep release rac_gui.tcl | grep -oE '\\b[0-9]{1,3}\\b'", property="release" ) ) # rac_gui_build_DEB.addStep(steps.ShellCommand( command=["mv", "rac_gui.tcl", "racgui"])) # rac_gui_build_DEB.addStep(steps.ShellCommand( command=["sed", "-i", "s+^set\ dir(lib)+set\ dir(lib)\ /usr/share/rac-gui/lib ;#+g", "racgui"])) # rac_gui_build_DEB.addStep(steps.ShellCommand( command=["sed", "-i", "s+\[pwd\]+/usr/share/rac-gui+g", "racgui"])) # rac_gui_build_DEB.addStep(steps.ShellCommand( command=["tar", "czf", util.Interpolate("../rac-gui_%(prop:version)s.orig.tar.gz"), "."])) # rac_gui_build_DEB.addStep(steps.ShellCommand( command=["dpkg-buildpackage"])) # rac_gui_build_DEB.addStep( steps.FileUpload( workersrc=util.Interpolate("../rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb"), masterdest=util.Interpolate("/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb") ) ) rac_gui_build_DEB.addStep( steps.MasterShellCommand( command=["/usr/local/bin/deploy-ftp.tcl", util.Interpolate("--local-file=/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb"), util.Interpolate("--remote-file=uploads/rac-gui/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb")] ) ) ############################################ # WIN ############################################ rac_gui_build_WIN = util.BuildFactory() rac_gui_build_WIN.addStep(steps.Git( repourl = 'https://bitbucket.org/svk28/rac-gui.git', haltOnFailure = True, submodules = True, mode='full', workdir='build', progress = True) ) # rac_gui_build_WIN.addStep( steps.SetPropertyFromCommand( command="powershell -command \"((Get-Content .\\rac_gui.tcl | Select-String -Pattern 'version:') -split '\\s')[2]", property="version" ) ) # rac_gui_build_WIN.addStep( steps.SetPropertyFromCommand( command="powershell -command \"((Get-Content .\\rac_gui.tcl | Select-String -Pattern 'release:') -split '\\s')[2]", property="release" ) ) # rac_gui_build_WIN.addStep(steps.ShellCommand( command=["c:\\bin\\freewrap.exe", "rac_gui.tcl"])) # , rac_gui_build_WIN.addStep(steps.ShellCommand( command=["c:\\Program Files\\7-zip\\7z.exe", "a", "-r", util.Interpolate("..\\rac-gui_%(prop:version)s-%(prop:release)s.win.zip"), "..\\build"])) # rac_gui_build_WIN.addStep( steps.FileUpload( workersrc=util.Interpolate("..\\rac-gui_%(prop:version)s-%(prop:release)s.win.zip"), masterdest=util.Interpolate("/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s.win.zip") ) ) # rac_gui_build_WIN.addStep( steps.MasterShellCommand( command=["/usr/local/bin/deploy-ftp.tcl", util.Interpolate("--local-file=/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s.win.zip"), util.Interpolate("--remote-file=uploads/rac-gui/rac-gui_%(prop:version)s-%(prop:release)s.win.zip")] ) ) c['builders'] = [ util.BuilderConfig(name="Rac-GUI-RPM-builder", workername='CentOS', factory=rac_gui_build_RPM), util.BuilderConfig(name="Rac-GUI-DEB-builder", workername='Debian-10', factory=rac_gui_build_DEB), util.BuilderConfig(name="Rac-GUI-WIN-builder", workername='Windows-10', factory=rac_gui_build_WIN), ] c['services'] = [] template=u'''\ <h4>Build status: {{ summary }}</h4> <p> Worker used: {{ workername }}</p> {% for step in build['steps'] %} <p> {{ step['name'] }}: {{ step['result'] }}</p> {% endfor %} <p><b> -- The Buildbot</b></p> ''' mailNotifier = reporters.MailNotifier(fromaddr="112@icvibor.ru", sendToInterestedUsers=False, mode=('all'), extraRecipients=["my@domain.local"], relayhost="mail.domain.local", smtpPort=587, smtpUser="buildbot@domain.local", smtpPassword="**********", messageFormatter=reporters.MessageFormatter( template=template, template_type='html', wantProperties=True, wantSteps=True)) c['services'].append(mailNotifier) c['title'] = "The process of bulding" c['titleURL'] = "http://vm-srv-build1:80/" c['buildbotURL'] = "http://vm-srv-build1/" c['www'] = dict(port=80, plugins=dict(waterfall_view={}, console_view={}, grid_view={})) c['db'] = { 'db_url' : "sqlite:///state.sqlite" }
Materials
In preparing the article, the following materials were used:
All Articles