Practical Glusto ExampleJonathan HollowayPrincipal Quality Engineer - Red Hat Storage

Practical Glusto Example

Very Brief Intro to Glusto

Glusto is...● Glusto is a “framework” or collection of commonly

used tools for developing scripts and tests for distributed systems.

● Primary components− A class combining a collection of tools in a single, easy-to-use interface.− Command-line wrapper to make dynamic config files available to test frameworks

● Built on open source and standard tools.− (e.g., Python modules, RPyC, SSH, REST, PyUnit, PyTest, Nose, etc.)

● Provides flexibility minus complexity during development.

− Simple import into existing scripts− Access to functionality via Python Interactive Interpreter− Code/Test/Code with IDE (e.g., Eclipse via PyDev)

Key Features● Remote calls via SSH● Remote calls via RPyC● Read and store config files in yaml, json, and ini formats● Logging● Templates● Provides a wrapper for unit test discovery and configuration● Works with multiple unit test frameworks (PyUnit, PyTest, Nose)● Simple REST client● Accessible via Python module, Python Interactive Interpreter, and a CLI

client.

Practical Glusto Example

Install Glusto

Installing Glusto● Installs via setuptools

− Directly from github with pip command− # pip install –upgrade \

git+git://github.com/loadtheaccumulator/glusto.git

− Via setuptools with python setup.py− # git clone https://github.com/loadtheaccumulator/glusto.git

# cd glusto# python setup.py install

● Docker container− docker.io/loadtheaccumulator/glusto

Installing glustolibs-gluster

● Installs via setuptools− Directly from github with pip command− # pip install –upgrade \

git+git://github.com/glusto-tests/glustolibs/glusto.git

− Via setuptools with python setup.py− # git clone https://github.com/gluster/glusto-tests.git

# cd glusto-tests/glustolibs-gluster# python setup.py install

● Docker container− Contains both Glusto and glustolibs-gluster

libraries− May be available by the time you see this

Practical Glusto Example

Write Tests

Test Scripts with Glusto

● Standard PyUnit, PyTest, or Nose format● Extendable via standard Python subclassing.● Glusto just plugs in via import without

dependency on Glusto to run basic tests.− from glusto.core import Glusto as g

● YAML config file containing the hosts● clients: [192.168.1.225]

servers: [192.168.1.221, 192.168.1.222, 192.168.1.223, 192.168.1.224]

Basic PyUnit Script Format

● import unittest

class MyTestClass(unittest.TestCase): @classmethod def setUpClass(cls): # runs before all test_ methods in the class

def setUp(self): # runs before each test_ method in the class

def test_example1(self): # test method representing a test case

def test_example2(self): # test method representing a test case

def tearDown(self): # runs after each test_ method in the class

@classmethod def tearDownClass(cls): # runs after all test_ methods in the class

Basic PyUnit Script Format

● import unittest

class MyTestClass(unittest.TestCase): @classmethod def setUpClass(cls): # runs before all test_ methods in the class

def setUp(self): # runs before each test_ method in the class

def test_example1(self): # test method representing a test case

def test_example2(self): # test method representing a test case

def tearDown(self): # runs after each test_ method in the class

@classmethod def tearDownClass(cls): # runs after all test_ methods in the class

Basic PyUnit Script Format

● import unittest

class MyTestClass(unittest.TestCase): @classmethod def setUpClass(cls): # runs before all test_ methods in the class

def setUp(self): # runs before each test_ method in the class

def test_example1(self): # test method representing a test case

def test_example2(self): # test method representing a test case

def tearDown(self): # runs after each test_ method in the class

@classmethod def tearDownClass(cls): # runs after all test_ methods in the class

Basic PyUnit Script Format

● import unittest

class MyTestClass(unittest.TestCase): @classmethod def setUpClass(cls): # runs before all test_ methods in the class

def setUp(self): # runs before each test_ method in the class

def test_example1(self): # test method representing a test case

def test_example2(self): # test method representing a test case

def tearDown(self): # runs after each test_ method in the class

@classmethod def tearDownClass(cls): # runs after all test_ methods in the class

Basic PyUnit Script Format

● import unittest

class MyTestClass(unittest.TestCase): @classmethod def setUpClass(cls): # runs before all test_ methods in the class

def setUp(self): # runs before each test_ method in the class

def test_example1(self): # test method representing a test case

def test_example2(self): # test method representing a test case

def tearDown(self): # runs after each test_ method in the class

@classmethod def tearDownClass(cls): # runs after all test_ methods in the class

import and class definition

● import unittestimport pytestfrom glusto.core import Glusto as gfrom gluster_base_class import runs_on

@runs_on([['distributed', 'replicated'], ['glusterfs', 'nfs']])class GlusterDemoTest(unittest.TestCase): @classmethod def setUpClass(cls): # runs before all test_ methods in the class

def setUp(self): # runs before each test_ method in the class

...

def tearDown(self): # runs after each test_ method in the class

@classmethod def tearDownClass(cls): # runs after all test_ methods in the class

setUpClass/tearDownClass

● ...

@runs_on([['distributed', 'replicated'], ['glusterfs', 'nfs']])class GlusterDemoTest(unittest.TestCase): @classmethod def setUpClass(cls): cls.masternode = g.config['servers'][0] cls.client = g.config['clients'][0] cls.mountpoint = '/mnt/gluster-mount'

g.run(cls.masternode, 'service glusterd start') g.run(cls.masternode, 'gluster volume start glustervol1') g.run(cls.client, 'mount /mnt/gluster-mount') ...

@classmethod def tearDownClass(cls): g.run(cls.client, 'umount %s' % cls.mountpoint) g.run(cls.masternode, 'gluster volume stop glustervol1') g.run(cls.masternode, 'service glusterd stop')

setUp/tearDown● ...

@runs_on([['distributed', 'replicated'], ['glusterfs', 'nfs']])class GlusterDemoTest(unittest.TestCase): @classmethod def setUpClass(cls): ... def setUp(self): self.filename = '%s/%s-%s' % (self.mountpoint, self.volume, self.mount) print('\nCreating file %s' % self.filename) g.run(self.client, 'touch %s' % self.filename)

...

def tearDown(self): print('Removing file %s' % self.filename) g.run(self.client, 'rm -f %s' % self.filename)

@classmethod def tearDownClass(cls): ...

test_ method(s)● ...

def setUp(self): ...

def test_mount_type(self): rcode, _, _ = g.run(self.client, 'mount | grep %s' % self.mount) self.assertEqual(rcode, 0, 'Mounted volume is not type %s' % self.mount)

@pytest.mark.bvt def test_create_file_with_touch(self): print('\nTesting file %s' % self.filename) rcode, rout, rerr = g.run(self.client, 'ls %s' % self.filename) self.assertEqual(rcode, 0, 'File does not exist')

def tearDown(self): ...

...

Practical Glusto Example

Run Tests

Run via /usr/bin/glusto● Glusto CLI wrapper provides config file(s) to tests

− $ glusto -c myconfig.yml --pytest='test_demo1.py'● PyTest provides xUnit style output

− $ glusto -c myconfig.yml \ --pytest='test_demo1.py --junitxml=results.xml'

● Accepts PyTest marker (-m) and filter (-k) parameters

− $ glusto -c myconfig.yml \ --pytest='test_demo1.py -m bvt'

$ glusto -c myconfig.yml \ --pytest='test_demo1.py -k mount'

Practical Glusto Example

Demohttps://asciinema.org/a/7sb4vox7ucasmgbwyeii4rheh

Info● Contact

− Email: gluster-devel@gluster.org− IRC: FreeNode #gluster-dev (loadtheacc)

● Libraries and Tests Repo− git clone http://review.gluster.org/glusto-tests− Mirrored at http://github.com/gluster/glusto-

tests● Glusto Repo and Docs

− http://github.com/loadtheaccumulator/glusto− http://glusto.readthedocs.io/