Update from-build-workgroup
-
Upload
michael-dawson -
Category
Technology
-
view
137 -
download
0
Transcript of Update from-build-workgroup
![Page 1: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/1.jpg)
Keeping the Node.js Community Infrastructure
HummingMichael Dawson, IBM
![Page 2: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/2.jpg)
About Michael Dawson Loves the web and building software (with Node.js!)
Senior Software Developer @ IBMIBM Runtime Technologies Node.js Technical Lead
Node.js collaborator and CTC member
Active in LTS, build, benchmarking , apiand post-mortem working groups
Contact me:
[email protected]: @mhdawson1https://www.linkedin.com/in/michael-dawson-6051282
![Page 3: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/3.jpg)
3
About The Build TeamLoves hardware, operating systems, infrastructure and building Node.js
Johan Bergström@jbergstroem
João Reis@joaocgreis
Rod Vagg@rvag
Alexis Campailla@orangemocha
Michael Dawson@mhdawson
Hans Kristian Flaatten@Starefossen
Julien Gilli@misterdjules
Rich Trott@trott
Myles Borins@thealphanerd
Ryan Graham @rmg
Imran Iqbal@iWuzHere
Phillip Johnsen @phillipj
![Page 4: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/4.jpg)
4
https://github.com/nodejs/build/blob/master/README.md
Providing computing infrastructure
Streamline testing and releasing
Operate and maintain
– Jenkins(ci and release)
– benchmarking
– nodejs.org
– and more
Purpose
![Page 5: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/5.jpg)
Preference for community donations
–Expands involvement
–In line with Node.js approach - contributions influence
direction
Cloud based
Spread across providers
5
Our Approach
![Page 6: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/6.jpg)
6
Infrastructure Providers
Tier 1
Tier 2
Microsoft Joyent IBM(osusol/softlayer)Voxer Scaleway NodesourceCloudFlare Arm
DigitalOcean Rackspace
![Page 7: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/7.jpg)
7
Community Donations
From -> https://github.com/nodejs/build/blob/master/README.md
Picture provided by Rod Vagg
![Page 8: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/8.jpg)
Jenkins Based
Separate Build and Release Instances
– https://ci.nodejs.org
– https://ci-release.nodejs.org
Build– 116 machines on last count
– 8 architectures (x86, x64, arm6, arm7, arm8, PPC be 64bit , PPC le
64 bit, s390x)
– 6 OS’s (Linux®, SmartOS, Windows®, OSX, AIX, FreeBSD)
Release– 14 machines on last count
– 8 architectures (x86, x64, arm6, arm7, arm8, PPC be 64bit , PPC le
64 bit, s390x)
– 4 OS’s (Linux, SmartOS, Windows, OSX)8
Infrastructure
![Page 9: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/9.jpg)
9
Many Linux Variants
Test
Release
![Page 10: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/10.jpg)
10
Builds
![Page 11: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/11.jpg)
Nothing too interesting here
bash like syntax
Most of logic built into:
11
Jobs
https://github.com/nodejs/node/blob/master/Makefile
![Page 12: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/12.jpg)
Mostly virtual machines except:
– Arm (Nodesource)
– Bare metal benchmarking machine (Softlayer)
Virtual machines
– free accounts from Infrastructure Providers
It’s a lot of Machines to manage ! (over 130)
Ansible - https://www.ansible.com/12
Machines
Picture provided by Rod Vagg
![Page 13: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/13.jpg)
Agentless
Connects over ssh
Key files
– ansible-inventory
– ansible-playbook.yaml
– ansible-vars.yaml
13
Ansible
https://github.com/nodejs/build/tree/master/setup
![Page 14: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/14.jpg)
14
Ansibleansible-inventory
ansible-vars.yaml
![Page 15: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/15.jpg)
15
ansible-playbook.yaml
![Page 16: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/16.jpg)
Multiple machines (at least 2 per platform)
ccache
Fanned jobs
Parallel test execution
16
Staying Nimble
Build/testacross all platforms
30 Mins
![Page 17: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/17.jpg)
Shared secrets
– Infra
– Test
– Release
17
Trust Delegation
![Page 18: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/18.jpg)
Website - https://nodejs.org
Benchmarking - https://benchmarking.nodejs.org/
Bots
Machine availability notifications
Backups
rsync mirrors
May support other projects
– Ex serialport module
18
Not just build/testing
![Page 19: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/19.jpg)
You may not have access to test on all platforms/os combinations
Potential access to build infra to test/investigate issues
https://github.com/nodejs/build/blob/master/doc/process/special_access_to_build_resources
.md
19
Machine Access
![Page 20: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/20.jpg)
OSX Support
Jenkins
20
Current Challenges
![Page 21: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/21.jpg)
Get Involved !
Great chance to learn
– Hardware, OS’s, Jenkins, Ansible
– Cloud provisioning (openstack etc)
Donate
Where
–Most work done through GitHub
issues/Pull Requests
–https://github.com/nodejs/build/
–IRC: #node-build at Freenode
![Page 22: Update from-build-workgroup](https://reader031.fdocuments.us/reader031/viewer/2022030309/58f22a831a28abac4e8b45b5/html5/thumbnails/22.jpg)
Copyrights and Trademarks
© IBM Corporation 2016. All Rights Reserved
IBM, the IBM logo, ibm.com are trademarks or registered
trademarks of International Business Machines Corp.,
registered in many jurisdictions worldwide. Other product and
service names might be trademarks of IBM or other companies.
A current list of IBM trademarks is available on the Web at
“Copyright and trademark information” at
www.ibm.com/legal/copytrade.shtml
Node.js is an official trademark of Joyent. IBM SDK for Node.js is not formally related to
or endorsed by the official Joyent Node.js open source or commercial project.
Java, JavaScript and all Java-based trademarks and logos are trademarks or registered
trademarks of Oracle and/or its affiliates.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.