Test Plan
Inkboard
Jonas Collaros
November 1, 2004
Table of Contents
· 7 Suspension Criteria and Resumption Requirements
· 12 Staffing and Training Needs
The software to be tested is a new feature to the Inkscape vector-based graphics drawing tool that allows the sharing of Inkscape documents with other users over a TCP connection. Communication is achieved via the Jabber and XMPP protocol for chat communication. These communication should be virtually transparent to the user- after the user chooses to connect a document to another user or group of users, the changes made by the other parties will simply appear. These changes will be detected internally to Inkscape and broadcast via the connected Jabber channel through formatted text messaging which are received internally to Inkscape and processed into similar changes on a recipient’s local, connected instance of Inkscape. The project is mostly concerned with making the whiteboard feature work reasonably well but is specifically not required to have a perfect system. The development community of Inkscape is expected to use our documents, methodologies, and suggestions to further the capabilities of the system according to their desires as they arise in the future.
This test plan will outline general testing guidelines rather than actual test cases.
Inkscape website: http://www.inkscape.org/
Inkboard website: http://inkboard.sourceforge.net/
Protocol definition: http://www.xmpp.org/specs/
To be tested is the first version of the whiteboard connectivity feature in Inkscape version 0.40. Test items include user-to-user and chat room document sharing, document synchronization and collision resolution. In the most common cases the inner workings of the chat should be unknown to the user.
Test cases that must be performed include those outlined in the Use Case Model, Vision Document and Supplemental Specification. Features of the Inscape product should be supported by the whiteboard extension except for a few cases, such as the undo tree. For details consult the listed documents.
Beyond basic regression testing, we will not be responsible for the testing of any Inkscape features in an offline setting that we are not responsible for implementing or have not changed with our added functionality. For features chosen not to be implemented, such as a local-undo in an actively-shared document or TLS and SASL security and encryption layers within the protocol, testing will simply include verifying the features are not present or blocked. Testing includes and is limited to any changes we make to the Inkscape version 0.40 code base and cursory inspection of other Inkscape features to ensure that no obvious side-effects have occurred.
The testing phase of development for our project will begin in the latter winter quarter, preferably as soon as a code-complete state can be reached. The code complete stage is scheduled for the end of the winter quarter, see the project schedule for details. Testing should be performed in a black box context, and individuals should test features that they have not themselves directly implemented. Given the small size of the team and the resultant difficulty in finding independent code sections which a member has not been involved with, this standard will be preferred but not requires. The Open Source nature of the project, the other developers of Inkscape will provide additional testing as developers who have not directly implemented the features we have created. This testing support will be available through the feedback of the Inkscape developer community during creation, and after the testing phase (during the maintenance phase). Sign-off on a feature will indicate an acceptable level of correctness and suitability verified on the feature, and as many known bugs as possible documented, even if not fixed, so that future Inkscape developers will be able to correct them. Sign-off on all features will indicate suitability for stable release.
While formal code inspections will not be required due to limited time, peer-understanding and high-level review of code is encouraged. Most development will follow an “extreme programming” style in which one or more additional developers are present to correct the work of the developer at the keyboard and to provide insight.
Unit testing will be outlined in several detailed test cases. Testers will be assigned to execute specific test cases over the units to be tested to verify the correctness and suitability of implemented features, including the verification of the proper functioning of all supported Inkscape features. Units will be committed to CVS repositories and declared “ready” only after the developer is reasonably certain that it is ready for more formal testing.
Testing will be done on Linux systems as well as a possible test passes on other platforms before release. Stress and load testing will be outlined in the test case document after a threshold or desired limits have been established. Stress and load testing is not the primary concern of this project. The project is mostly concerned with making the whiteboard feature work reasonably well but is specifically not required to have a perfect system. The development community of Inkscape is expected to use our documents, methodologies, and suggestions to further the capabilities of the system according to their desires as they arise in the future.
All preexisting Inkscape features that will be in some way changed or modified by the new whiteboard functionality (e.g. disabling of the undo feature when sharing) should be regressed to assure they perform properly in the “normal” or not-shared state. A courtesy “not-broken” test pass will be performed on preexisting Inkscape features not directly affected by our implementation. Detailed regression testing of the not-shared state of the system will not be required of this project given the short span of the project and the large capabilities of Inkscape.
Acceptance
testing will be performed as outlined above and in the test cases. After all
test cases have been run and all features signed-off on, the implementation
will be ready for acceptance by Ted Gould and other members of the Inkscape
development and user communities. Acceptance testing will occur throughout the
spring quarter via at least two releases to the Inkscape communities.
If the execution of a test case results in incorrect behavior
or behaves in an unexpected, unfriendly or disruptive manner, the items
involved in the behavior will have failed testing. The severity of failure and
the priority of fixing the particular items depend on the magnitude of the
incorrect or undesired behavior. A passing feature is one that can be
signed-off on. Sign-off on a feature will indicate an acceptable level of
correctness and suitability verified on the feature, and as many known bugs as
possible documented, even if not fixed. Bugs should not be rampant, but minor
issues or well-documented and reasoned major issues are allowed. The project is
mostly concerned with making the whiteboard feature work reasonably well but is
specifically not required to have a perfect system. The development community
of Inkscape is expected to use our documents, methodologies, and suggestions to
further the capabilities of the system according to their desires as they arise
in the future.
7 Suspension Criteria and
Resumption Requirements
When testing a given feature partition of the implemented whiteboard system, testing should continue until the tester has reached a “blocked” state, namely, when the presence of bugs prohibits testing beyond what has already been verified by the tester. In this case, the tester must suspend testing of the features until fixes or other resolutions to the code are available, and then resume. It is expected that the testing will go though several iterations of this testing, fixing, testing stage given the limited resources of the project and the resultant lack of a test team separate from the development team. This is in some ways mitigated by the open source nature of the product, in that the Inkscape developers can help to locate bugs, but we will be required to test even if the Inkscape development community chooses to focus on other matters.
Once all features have been signed-off, testing is then complete, though bug fixes may be (and are expected to be) made post-release, especially given the multi-release format of the project.
Deliverable documents include a formal test case document outlining exactly which test cases will be performed or will be required to be performed to ensure the acceptability and thorough verification of the various aspects of the whiteboard features. Test results will be in the form of ‘”signed-off” test cases and documented bugs both resolved and pending. Incrementally better versions of releases, at least two, will also be a product of testing.
To prepare for and perform testing, first, a rigorous test case document must be composed that covers each feature that should be verified to ensure correctness of our code and the code we modify and interact with. Afterwards, the code will need to reach a “testable” status, preferably upon code-complete. Coders should have the resources necessary to being testing the developed code immediately thereafter. Maintenance testing will be handled in the final term post release to resolve unforeseen issues and achieve stability
Task 1- User Connection
The user will be able to connect to a jabber instant messenger name (registered outside of Inkscape). This user should appear available to all other Jabber clients subscribed to that user. The users that the account is subscribed to should appear in a buddy list generated by the program. This list does not need to refresh automatically, just on demand. Messages sent accidentally from outside of the Inkscape environment should be ignored by the program. This will be tested through multiple logins.
Task 2- One-to-One Document Sharing
The user will request connection with another user to whom he is subscribed. This will create a new resource connected to the jabber name to associate with Inkboard messages. Direct connections messages should pass back and forth freely along these established channels, triggering resultant changes in each local instance Inkscape opened on each user’s computer. The user should not see these messages, only the results. Document sharing will be tested constantly though simultaneous modification of a directly shared document.
Task 2.1- User Dropouts (one-to-one)
When a user drops out or is disconnected from the Jabber server, the other user engaged in a whiteboard session should be notified via a popup window. Disconnections should not have any other adverse effects on the program. Dropouts will be simulated by quitting the Inkscape program unexpectedly.
Task 2.2- User Timeouts (one-to-one)
When a user times out or is disconnected from the Inkscape document, the other user engaged in a whiteboard session should be notified via an unobtrusive message flash at the bottom of the screen. The timed out user should be presented with notification of the timeout and the option to reconnect to the document and receive a fresh update of the whole document. Timeouts should not have any other adverse effects on the program. Timeouts will be simulated by connection through very slow jabber servers prone to document timeouts.
Task 2.3- Document Updates (one-to-one)
Updates of the whole document should not cause any unexpected behavior. Document updates should delete the old version of the document on the updating user’s Inkscape instance and replace it with an identical version of the current document on the directly-connected user’s machine. Document updates will be tested primarily by connecting to new sessions (which trigger document updates) and by testing timing out of users.
Task 3- Chatroom Document Sharing
The user will request connection to a Jabber chatroom containing zero or more other users. This will create a new resource connected to the jabber name to associate with Inkboard messages. Direct connections messages should pass back and forth freely along these established channels, triggering resultant changes in each local instance Inkscape opened on each chatroom connected user’s computer. The user should not see these messages, only the results. Document sharing will be tested constantly though simultaneous sharing and modifications of a document in a chatroom setting with various numbers of other Inkscape users.
Task 3.1- User Dropouts (chatroom)
When a user drops out or is disconnected from the Jabber server, the other users engaged in a chatroom whiteboard session should be notified via an unobtrusive message flash at the bottom of the window. Disconnections should not have any other adverse effects on the program. Dropouts will be simulated by quitting the Inkscape program unexpectedly.
Task 3.2- User Timeouts (chatroom)
When a user times out or is disconnected from the Inkscape document, the other users engaged in the chatroom whiteboard session should be notified via an unobtrusive message flash at the bottom of the screen. The timed out user should be presented with notification of the timeout and the option to reconnect to the document and receive a fresh update of the whole document. Timeouts should not have any other adverse effects on the program. Timeouts will be simulated by connection through very slow jabber servers prone to document timeouts.
Task 3.3- Document Updates (chatroom)
Updates of the whole document should not cause any unexpected behavior. Document updates should delete the old version of the document on the updating user’s Inkscape instance and replace it with an identical version of the current document. The current document in a chatroom is selected by requesting a whole document send from the fastest connection in the chatroom. Further changes since this “current” version will be logged and stored on the updating user’s change queue and implemented independently once the document is received. Document updates will be tested primarily by connecting to new sessions (which trigger document updates) and by testing timing out of users.
Task 4- Collision Detection
No two users should be allowed to create the same new object with the same unique identifier. Such instances will result in a renaming of the offending object(s). Other changes, such as two users choosing to move an object to different places, will be handled normally (whoever enacts the change latest will have the standing location). Collisions will be simulated through specific usage of messages using the same object id or message change number. Collisions will also be simulated through the simultaneous physical change of objects via the Inkscape program.
To test this feature, we will need to be able to replicate typical user environments. This involves user-to-user and chat room based XMPP sessions over a TCP connection. Simulated collisions and lag can also be used for more rigorous testing.
While the coder of a feature should make reasonable efforts to ensure submitted code is acceptable in implementation and should be quick to resolve found bugs in his or her code, the ultimate responsibility for the correctness of a feature lies with the assigned tester. The tester of a feature should work with the coder to bring the code to an acceptable state, whereupon the tester will “sign-off” on the feature. The tester and coder of a given feature may be the same person in certain cases.
Also, while support of the Inkscape developer community with be of benefit, our team should not rely on such feedback for adequate testing of the whiteboard feature.
12 Staffing and Training Needs
Testing will be performed by the developers before release,
who should be familiar with the target uses and systems, so this item is not
applicable.
To prepare for and perform testing, the code will need to reach a “testable” status, preferably upon code-complete. The testing phase of development for our project will begin as early as the latter winter quarter, or as late as early spring quarter. Coders should have the resources necessary to begin testing the developed code immediately thereafter. Maintenance testing will be handled in the final term post release to resolve unforeseen issues and achieve stability. A schedule for the testing and correction of known bugs is available via the Project Schedule artifact, outlined under the Spring Quarter.
A high-risk to proper testing is lack of time. If the implementation and coding stages of development run too long, adequate time may not be available for complete and thorough testing. To avoid this, testing must begin as soon as possible on portions of code that have been completed, preferably immediate after a code complete date.
We must obtain approval from Ted Gould and the members of the Inkscape development and user community in general.
|
# |
Who |
What |
|
1 |
All |
Test case document |
|
2 |
All |
Revisions |
|
Date |
Who |
Revision |
|
11/9/2004 |
Jonas |
First document version |
|
11/11/2004 |
Jonas |
Document update: added details and elaborations |
|
|
Brandi |
Updated document and added outlines for tasks in section 9 |