Linaro Android

Make a list of tests present in AOSP source

Registered by Zygmunt Krynicki

Why:
We want to track which tests are in present in AOSP and if we have them built into our images. To make that possible we need to construct (and possibly maintain over time) a list of tests that are present in particular git repositories.

Who:
Zygmunt Krynicki (zyga)

Where:
The scripts that compute this, as well as the actual data, is stored in git at https://github.com/zyga/android-test-tools. Once the blueprint is nearing completion I will export that data to google spreadsheet for easier use.

Context:
The testing card

Blueprint information

Status:
Complete
Approver:
Zach Pfeffer
Priority:
Medium
Drafter:
Zygmunt Krynicki
Direction:
Approved
Assignee:
Zygmunt Krynicki
Definition:
Approved
Series goal:
Proposed for 2011q2
Implementation:
Implemented
Milestone target:
milestone icon 12.07
Started by
Zygmunt Krynicki
Completed by
Zach Pfeffer

Related branches

Sprints

Whiteboard

[2012-07-05 zkrynicki] Working on a script that analyzes source and build tree. I have two builds that differ only by the TARGET_BUILD_VARIANT to inspect.
[2012-07-05 zkrynicki] Create a small repo to keep my scripts at https://github.com/zyga/android-test-tools
[2012-07-05 zkrynicki] Initial list of extra test apks: https://github.com/zyga/android-test-tools/blob/master/extra-apks-panda.txt
[2012-07-06 zkrynicki] Initial list of extra test executables (linux binaries): https://github.com/zyga/android-test-tools/blob/master/extra-execs-panda.txt
[2012-07-06 zkrynicki] Found a few interesting directories that should be considered: native{benchmark,test,stresstest} and benchmark (all in the data directory). Also found skia tests (a standalone binary) some EGL tests (standalone binaries).
[2012-07-06 zkrynicki] A few ideas as I EOD: 1) scan the code for c/java files with the word 'test' in the pathname. Scan the build log to see if any are missing. I suspect that there are large 'islands' of tests that are special/not enabled anyway. I'll try to write some of that next week.
[2012-07-18 dzin] please add Headline and Acceptance fields.
[2012-07-23 zkrynicki] Preparing to close the blueprint. Removed output and description of a gtest-based test output and linked to a dedicated blueprint
[2012-07-23 pfefferz] Test lists and a script to compare builds is here:
<zyga> https://github.com/zyga/android-test-tools/blob/master/extra-execs-panda.txt
<zyga> https://github.com/zyga/android-test-tools/blob/master/extra-apks-panda.txt
<zyga> https://github.com/zyga/android-test-tools/blob/master/compare-builds.py
<zyga> https://github.com/zyga/android-test-tools

Quick links to results:
https://github.com/zyga/android-test-tools <- sorry, no direct links, github uses commit ID in URLs

Headline: A list of tests available in AOSP has been produced
Acceptance: A list of tests available in AOSP has been produced

(?)

Work Items

Work items:
Discover out/target/product/*/data/testinfo/test_defs.xml: DONE
Compare test to non-test build: DONE
Compile a list of new .apks: DONE
Filter incorrectly classified apks: DONE
Include native{test,benchmark,stresstest} and benchmark: DONE
Compile a list of new executables: DONE
Include existing executables that look like tests: DONE
Store all of that in a git repo: DONE

Dependency tree

* Blueprints in grey have been implemented.

Nearby

This blueprint contains Public information 
Everyone can see this information.