Hola:
Aprovechando que se están convocando las Jornadas "Sólo Pruebas 2006" y
las JTS 2006 de Valencia, viene al dedo este mensaje de Shrikant Gavali,
desde la India, que referencia varios articulos y libros para decidirse
sobre el uso o no de herramientas para automatización de pruebas.
Tambien resuelve dudas sobre tipos de pruebas, tipos de herramientas, etc.

Un saludo,
Ramiro Carballo.
GESEIN.

----------------------------

Q: How do I know when to Automate?

A: In a survey reported in August 2005
(www.methodsandtools.com/facts/facts.html), two out of every five
organisations that had invested in test automation tools admitted to not
using them. The question wasn't asked, but it's likely that a
substantial proportion of the other three fifths weren't getting much
value out of their investment.

Test automation isn't a magic cure-all for everything that's ailing in
our test activities. Testing is a rather chaotic activity in most
software shops, and automating it runs the risk of turning it from slow
chaos to high-speed (and much more destructive) chaos. See the two
following articles:

"Test Automation Snake Oil" (James Bach,
http://www.satisfice.com/articles/test_automation_snake_oil.pdf )

"Automation Myths" (M. N. Alam,
http://www.benchmarkqa.com/PDFs/automation_myths.pdf )

Is your organisation ready to automate? This article will help you decide:

"When Should a Test Be Automated?" (Brian Marick,
http://www.stickyminds.com/r.asp?F=DART_2010 )

There are several related major reasons why test automation ware often
becomes shelfware within a period of about three months, but the
commonest is lack of technical readiness for the minute attention to
detail that test execution tools mostly require. Closely related is the
need for financial support to provide this technical readiness; in the
Methods and Tools survey mentioned above, the commonest reason for
dropping automation tools was that "managers had the budget to buy the
tools, but did not consider the budget and time needed to train people
in the tools and use them in real projects."

In any case, test execution tools are not necessarily the best place to
start. Bringing testing under good administrative control with a "test
management tool" -- one that helps manage the configuration and status
relationships between specifications (or software features) and tests --
may be a better starting point. The following book reviews automation
assistance for every major testing activity, from the Business Analysis
phase to the Test Program Review and Assessment Phase. It's also an
excellent choice for an organisation wanting to bring its testing under
good control before undertaking automation (in fact, it's one of the
best books available on organising the test function, automated or no):

Automated Software Testing: Introduction, Management, and Performance
(Elfriede Dustin, Jeff Rashka, and John Paul).


Q: What tool should I use to automate my application or web site?

A: After you've determined that you're ready to automate, you need to
figure out what you need your tool to do, and what features will make it
beneficial. Some things to look for are discussed here, though both
these resources are mainly concerned with test execution tools, rather
than test planning, design, or management tools:

"Software Test Automation and the Product Life Cycle: Implementing
software test in the product life cycle" (Dave Kelly,
http://www.mactech.com/articles/mactech/Vol.13/13.10/SoftwareTestAutomation/
)

"Making the Right Choice: The features you need in a GUI test automation
tool" (Elizabeth Hendrickson, http://www.stickyminds.com/r.asp?F=DART_5092.
here is a more recent version of this article in the STQE (now Better
Software) special edition, The 2004 Tools Directory.)

Having created a list of required benefits and matching features, go on
to evaluate specific products that you think will fit the bill based on
those requirements. Here are some test automation tools lists:

http://www.testingfaqs.org/
http://www.opensourcetesting.org/
http://www.qalinks.com/Tools/Automated_Testing/
http://www.aptest.com/resources.html
http://www.softwareqatest.com/qattls1.html

The Software Test Tool Evaluation Centre (
http://test-tools.technologyevaluation.com/ ) provides numerous white
papers, and a free research and evaluation service (to be found at:
http://tinyurl.com/lsyfj )

The following articles compare some of the most popular commercial
automation tools:
http://www.automationexpertise.com/Files/SilkWrFeatures.pdf
http://www.geocities.com/walter_gorlitz/Software_Test/
(AppB.pdf and automation_tools.doc)

These articles are also good reads on the topic:
http://www.benchmarkqa.com/PDFs/automation_myths.pdf
http://www.satisfice.com/articles/test_automation_snake_oil.pdf

You could also look at the SQATester home site resources at
http://www.sqatester.com/tools/index.htm

A step-by-step process for evaluating test tools, running a pilot for
the tested tool, implementing it across the organisation, and making
good use of it now you've got it, is found in the book:

Software Test Automation: Effective use of test execution tools
(Mark Fewster and Dorothy Graham).

Chapter 10 of the book, "Choosing a tool to automate testing", including
forms and checklists, is summarised on-line at:
http://www.grove.co.uk/Tool_Information/Choosing_Tools.html


Q: What are Load, Stress, Volume, etc. testing?

A: Volume testing is a way to test functionality. Stress testing is a
way to test reliability. Load testing is a way to test performance.

Testing an application under heavy but expected loads is known as load
testing. It generally refers to the practice of modeling the expected
usage of a software program by simulating multiple users accessing the
system's services concurrently. As such, load testing is most relevant
for a multi-user system, often one built using a client/server model,
such as a web server tested under a range of loads to determine at what
point the system's response time degrades or fails. Although you could
perform a load test on a word processor by or graphics editor forcing it
read in an extremely large document; on a financial package by forcing
to generate a report based on several years' worth of data, etc.

When the load placed on the system is accelerated beyond normal usage
patterns, in order to test the system's response at unusually high or
peak loads, it is known as stress testing. Stress testing is often
incorrectly used interchangeably with load and performance testing.

In a stress test, the load is usually so great that error conditions are
the expected result, although there is a grey area between the two
domains and no clear boundary exists where you could say that an
activity ceases to be a load test and becomes a stress test.

There is little agreement on what the specific goals of load testing
are. The term is often used synonymously with performance testing,
reliability testing, and volume testing.

Performance testing is usually performed to determine how fast some
aspect of a system performs under a particular workload. It can serve
different purposes: to demonstrate that the system meets performance
criteria; to compare two systems to find which performs better; to
measure what parts of the system or workload cause the system to perform
badly.

In the diagnostic case, we use tools such as profilers to measure what
parts of a device or software contribute most to the poor performance.
In performance testing, it is often crucial (and often difficult to
arrange) for the test conditions to be similar to the expected actual use.

A reliability test determines how likely a piece of hardware (or
sometimes software) is to fail. It is part of reliability theory, used
originally as a tool to help nineteenth century insurance companies
compute profitable rates to charge their customers. Statistical models
appropriate for any of these are generically called "time-to-event"
models. Death or failure is called an "event", and the goal is to
project or forecast the rate of events for a given population or
probability of an event for an individual.

Volume testing is used to determine if the system under test can handle
the required amounts of data, user requests, etc.

Soak testing occurs when running a system at normal to high levels of
load for prolonged periods of time. A soak test would normally execute
several times more transactions in an entire day than would be expected
in a busy day. This should identify any performance problems that appear
after a large number of transactions have been executed.

For some of the basics on this type of testing, please read the
following series:

Web Page Response Time 101
http://www.stickyminds.com/r.asp?F=DART_5030

Web Load Test Planning
http://www.stickyminds.com/r.asp?F=DART_5084

The Science and Art of Web Site Load Testing
http://www.stickyminds.com/r.asp?F=DART_1939

Getting Started : Stressing Web Applications Stress Early -- Stress Often
http://www.amibug.com/getinfo.shtm?present=webstress

http://www.perftestplus.com
http://www.performancetester.com


Shrikant gavali
SQA Engineer





______________________________________________
LLama Gratis a cualquier PC del Mundo.
Llamadas a fijos y móviles desde 1 céntimo por minuto.
http://es.voice.yahoo.com