Test Case Management Core

[2012-11-26, stylesen]

Documentation for the new format of pulling test definition from repositories is getting added as part of branch - https://code.launchpad.net/~stylesen/lava-dispatcher/testdef-from-repo-doc

[2012-11-21, stylesen]

Merged lp:~stylesen/lava-dispatcher/testdef-from-repo

[2012-11-20, stylesen]

Code changes done as per review comments from mwhudson. Following are notable ones:

    1) Define the parameter schema for job which specifies the test definitions.
    2) Refactor _get_test_definition_from_*, which now got merged into
       _get_test_definition
    3) Use subprocess.check_call instead of subprocess.call
    4) Fix some indentation.

[2012-11-19, stylesen]

MP submitted for branch lp:~stylesen/lava-dispatcher/testdef-from-repo

[2012-11-16, stylesen]

Done with the following at revision 449.
    1) Do not use '.has_key()' on a dictionary, use 'in' instead.
    2) Remove the counter for directory naming, use timestamp value instead.

Pending:
   Still confused with the cloning repo directly on target mount point :(

[2012-11-15, stylesen]

Implemented review comments from Andy in revision 448.

[2012-11-12, stylesen]
                                            ***** UPDATE 3 *****

Revision 446 is pushed, which brings in the following:

 * Adds support for pulling test definitions from bzr branches.

                                            ***** UPDATE 2 *****

Revision 445 is pushed to the same branch. But this time we have the following:

 * Support defining multiple git repositories in the job file where test definitions are found.
 * There could be multiple test definition yaml files within the same repository which could be defined in the job definition and that will be pulled and executed.
 * If a testdef.yaml file depends on many scripts those could be part of the repository and pulled in for executing the testdef.yaml
 * Still there is no support for bzr.
 * parameter_schema has changed as follows:

    parameters_schema = {
        'type': 'object',
        'properties': {
            'testdef_urls': {'type': 'array', 'items': {'type': 'string'},
                             'optional': True},
            'testdef_repos': {'type': 'array', 'items': {'type': 'object'},
                              'optional': True},
            'timeout': {'type': 'integer', 'optional': True},
            },
        'additionalProperties': False,
        }

                                            ***** UPDATE 1 *****

Initial version (r444) of supporting test definitions is available in - https://code.launchpad.net/~stylesen/lava-dispatcher/testdef-from-repo
This version supports the following:
 * Supports git repos only
 * Also supports testdef_urls as before
 * Job definition Format is subject to change, in future versions
 * paramter_schema as of now:

    parameters_schema = {
        'type': 'object',
        'properties': {
            'testdef_urls': {'type': 'array', 'items': {'type': 'string'},
                             'optional': True},
            'testdef_repo': {'type': 'string', 'optional': True},
            'testdefs': {'type': 'array', 'items': {'type': 'string'},
                         'optional': True},
            'timeout': {'type': 'integer', 'optional': True},
            },
        'additionalProperties': False,
        }

[2012-11-06, doanac] do we want to try and do repo caching logic for this milestone? I'm thinking this is a 12.12 item

TODOS:
For new testdef format we need to think about including new things such as:
 * meta-data: targetted for lava-android-test, lava-test, lava-test-shell
 * meta-data: OS like ubuntu, android, oe
 * meta-data: device-types like panda, origen, etc
 * overall test description
 * descriptions of test cases, maybe think about some type of nesting logic for optional test cases

Meta:
Headline: Initial work on test case management has begun
Acceptance: LAVA can execute bigLITTLE testing using new YAML testdef and bzr/git repository
Roadmap id: CARD-200

Work items:
convert current lava-test-shell JSON testdef to YAML: DONE
add ability to pull a bzr/git repo that includes the test def: DONE
add ability to pull a specific revision of the repo: DONE
document/define new YAML format: DONE