Shared Object images in Docker: What you need is what you want.
32
Reduce, Reuse, Recycle: Repeatable library images for Docker. Steven Lembark Workhorse Computing [email protected]
-
Upload
workhorse-computing-llc -
Category
Technology
-
view
352 -
download
1
Transcript of Shared Object images in Docker: What you need is what you want.
Reduce, Reuse, Recycle: Repeatable library images for Docker.
Steven LembarkWorkhorse [email protected]
Distributing Docker Images
Stock doc's use Ubuntu distro image.
Includes systemd, among other things.
Bloated
Distributing Docker Images
Why distro?
/bin?
/usr/bin?
/sbin?
/var/spool?
Distributing Docker Images
Why distro?
Shared Object Lib's (.so).
Distributing Docker Images
Why distro?
Shared Object Lib's (.so).
Your executable + Ubuntu == runnable.
Distributing Docker Images
Why distro?
Shared Object Lib's (.so).
Your executable + Ubuntu != runnable.
Except when it isn't.
Docker is not a VM
Execuatables use versions of lib's.
Not all versions are [bug-] compatible.
RH/SuSE/Arch/Gentoo + Ubuntu == ???
libc differences are the worst.
What you need are your libs.
Q: How to distribute?
One approach: Dup /lib*, /usr/lib* .
4 /lib32
38 /lib64
372 /usr/lib32
2526 /usr/lib64
About 3GB on my notebook.
Development server? (ouch)
What you need is all you want
Q: What do you need?
What you need is all you want
Q: What do you need?
A: Ask.
ldd lists SO paths.
What you need: ask ldd
# Path to .so and virtual offset of entry point.
$ ldd /opt/bin/perllinux-vdso.so.1 (0x00007ffee215e000)libperl.so => /opt/perl/5.22/lib/5.22.0/x86_64-linux/CORE/libperl.so (0x00007f6b40026000)libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6b3fe0a000)libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f6b3fbf2000)libdl.so.2 => /lib64/libdl.so.2 (0x00007f6b3f9ee000)libm.so.6 => /lib64/libm.so.6 (0x00007f6b3f6f2000)libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f6b3f4bb000)libutil.so.1 => /lib64/libutil.so.1 (0x00007f6b3f2b8000)libc.so.6 => /lib64/libc.so.6 (0x00007f6b3ef1d000)/lib64/ld-linux-x86-64.so.2 (0x00007f6b40417000)
What don't you need?
Then again, not everything needs shared lib's:
$ ldd /opt/perl/lib/5.22.0/NEXT.pmldd: warning: you do not have execution permission for `/opt/perl/lib/5.22.0/NEXT.pm' not a dynamic executable
Q: How do you ask for what you need?
More than one way...
1. Scan it all.
Scan all of perl5's lib.
ldd everything.
filter out rejects.
uniq the rest.
Good: One image for everything.
Bad: Big (?)
2. Scan current
Construct docker's source directory.
Scan that with ldd.
App From "-libs" image.
Good: Small.
Bad: More work.
2. Scan current
Construct docker's source directory.
Scan that with ldd.
App From "-libs" image.
Good: Small.
Bad: More work.
Good: Easy to automate.
Q: Which PM's do I need?
A: Ask perl.
%INC has all of your module paths.
x \%INC0 HASH(0x24d8070) '/opt/perl/5.22/lib/site_perl/5.22.0/x86_64-linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix' => '/opt/perl/5.22/lib/site_perl/5.22.0/x86_64-linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix' 'AutoLoader.pm' => '/opt/perl/5.22/lib/5.22.0/AutoLoader.pm' 'B.pm' => '/opt/perl/5.22/lib/5.22.0/x86_64-linux/B.pm'
Q: Which PM's do I need?
A: Ask perl.
%INC has all of your module paths.
Hard to export for arbitrary program. x \%INC0 HASH(0x24d8070) '/opt/perl/5.22/lib/site_perl/5.22.0/x86_64-linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix' => '/opt/perl/5.22/lib/site_perl/5.22.0/x86_64-linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix' 'AutoLoader.pm' => '/opt/perl/5.22/lib/5.22.0/AutoLoader.pm' 'B.pm' => '/opt/perl/5.22/lib/5.22.0/x86_64-linux/B.pm'
So: Bulk is it.
cd /opt/perl;
find -H lib bin -type f |
xargs ldd 2>/dev null |
grep '.so' | grep -v ':$';
Menagrie of sharing
linux-vdso.so.1 (0x00007ffe8d3f6000)libc.so.6 => /lib64/libc.so.6 (0x00007ff244e37000)/lib64/ld-linux-x86-64.so.2 (0x00005622e44ad000)linux-vdso.so.1 (0x00007fffe4075000)libc.so.6 => /lib64/libc.so.6 (0x00007f92f978f000)/lib64/ld-linux-x86-64.so.2 (0x00005622e63a1000)
Menagrie of sharing
linux-vdso.so.1 (0x00007ffe8d3f6000)
Q: Why no path?A: This is a kernel binding.
No file on disk.
Menagrie of sharing
linux-vdso.so.1 (0x00007ffe8d3f6000)
Q: Why no path?A: This is a kernel binding.
No file on disk.
Be careful what you search for!
ldd uses fat commas
find -H /opt/perl/lib |
xargs ldd 2>/dev/null |
grep '=>' | cut -d ' ' -f3|
sort -d | uniq ;
ldd uses fat commas
Except when it doesn't:/lib64/libresolv.so.2/lib64/librt.so.1/lib64/libtinfo.so.6/lib64/libutil.so.1/lib64/libz.so.1not/usr/lib32/libatk-1.0.so.0/usr/lib32/libbz2.so.1/usr/lib32/libcairo.so.2
cpio copies paths
Copy to build dir.
cpio ignores "not", keeps going.
#!/bin/bash
cd $(dirname $0);
dirs='/opt/perl/{lib,bin}';
find $dirs | xargs ldd | cut | sort | uniq |
cpio -pdv . ;
Result:
$ ls -a -1.dockerignorebuildDockerfilelib32lib64usr
$ du -msx; # compare to 3000MB!8
.dockerignore
One line: build
Command line
$ docker build --rm . \
http://host:5000/lembark/perl-so;
Not so bad.
build script
#!/bin/bash
cd $(dirname $0);name=$(basename $(dirname $PWD));
find -H /opt/perl/{bin,lib} -type f |xargs ldd 2>/dev/null | grep '=>' | cut -d' ' -f3 |uniq | sort -d | uniq |cpio -pd . ;
docker build --rm . \http://localhost:5000/$(whoami)/$name;
perl-exec from perl-so
FROM .../lembark/perl-so
All the libs you need.
In one place.
Summary
Code requires SO's.
If SO's match things work.
Get what you need using ldd.
Dockerize perl from SO image.
Result: Truly Lazy Docker.
