Frequently Asked Questions

We've been asked a lot of questions about our service, from the why to the how, and we think they're worth sharing:

Why should I use your service?

As developers, we become "blind" to problems in our application, problems that limit our software sales. It's not enough to have a solid design or to pass the Joel Test. Developers are humans, and humans develop a psychological blindness to to problems in their software.

(Did you see that? The double "to" in the previous sentence? That's why writers have copy editors and why you should have an independent software review.)

Our goal is to help you improve your software by thoroughly, honestly, and objectively evaluating it. We'll offer targeted suggestions that we guarantee will help you improve your software.

How does your service work?

It's very simple:

Place a review order
We contact you with a schedule of delivery and you prepay for services rendered
We then download and install (or access via the web, if hosted) your software
We review your software, soup to nuts
We send you (via e-mail) our completely honest and thorough evaluation
If you are not completely convinced that our report found problems that would have cost you money, contact us and we'll refund your money, no questions asked

How much does your service cost?

Only $500 (US), per application, per instance. This means if you want us to review your "Hello World 1.0", then your "Hello World 1.1" programs, you will be charged for each review. Similarly, if you want us to review your "Hello World 1.0" and "Goodbye World 1.0" programs, you will be billed for each review.

If your application is exceptionally robust, we may require a custom price quotation. This is rare. Consider the Windows calculator, Microsoft Excel, and Intuit QuickBooks. All deal with number crunching, but the complexity increases with each. To achieve the same evaluation depth, the most complicated application requires more time investment and, therefore, a custom price quotation.

How long does it take for you to review my software?

About 1 business week after receiving payment, depending upon the complexity of the application and our current workload.

I can't wait "about 1 business week"; I need my review now!

We offer expedited review service of 2-3 days, on a case-by-case basis. An additional fee of $250 will be assessed. When we speak, simply let us know that you need a "rush" job, and we'll adjust the schedule and cost accordingly.

What types of software do you evaluate?

If it runs on a computer and has more than 1 user, we'll review it: desktop applications, web-based applications, web sites, hand-held/mobile applications, embedded applications, etc. Our team's experience spans a wide range of software and UI environments, enabling us to provide targeted recommendations for a diverse set of software applications.

What aspects do you review?

Basically everything from the grand scheme to the tiny details. We look at:

The installation process
The purported functionality
The documentation (printed, inline, and context-sensitive)
The packaging (if applicable)

And we are looking for:

Overall consistency
Ease of learning
Operates without error
Highly usable, regardless of user disability
Robust even in unpredictable environments
Gracefully degrades in fatal conditions
Tolerates user inexperience
Appropriately secure
Has acceptable performance
Tidiness of operation
Aesthetically pleasing

What aspects do you not review?

We're not coverage or stress testing your application, but we will document any noticeable performance problems.

We are not deliberately trying to crash or compromise your software, but we will poke it in ways that typical users might. This means, for example, we'll try to use last names containing apostrophes, but we won't try SQL injection attacks.

What's in the final report?

A thorough and completely honest evaluation of your software application, including:

A description of our test environments and procedures.
Our understanding of the user group(s) who will use your software (inexperienced/experienced, young/old, etc.).
A summarization of what we found worked well.
An analysis of show-stoppers and major problems (in our estimation), including reproduction instructions.
Lists of problems encountered, our estimation of their severity, and possible solutions (including any workarounds we found).
Our recommendations for any additional, necessary reviews.

Think of the final report as a list of improvement ideas. We recognize that the finest set of recommendations will be rejected if the form is hostile, so we deliberately speak objectively and factually, while remembering and respecting that you have invested substantial time in this software.

The report is written in American English. We avoid using American idioms that wouldn't be understood outside of the U.S. We do, however, occasionally use "standard" jargon.

What if I have questions after receiving the final report?

Please contact us! We want to ensure that you fully understand our review and are completely satisfied by our work.

However, when you contact us, please do so with a list of specific questions that we can answer. We are not able to rehash the report or provide any additional presentations.

My software requires installation. Which OS do you support?

We support a variety of systems:

Microsoft Windows 2000 and all releases since (up to and including Vista)
Apple OS X
Linux Gentoo, Fedora Core, RedHat, Debian, SuSE, and Mandrake
A few other esoteric systems (like IRIX), so let us know

Be warned, however, that we will try your install:

on every platform you claim to support, and
on sterile systems as well as systems polluted with every shiny software gizmo we could find to install.

Do you keep my software installed after testing?

Nope, it's wiped out. Our review environment consists of pre-configured sandboxes in a virtual machine. When we conclude our review, we wipe out the installation.

Isn't this just outsourced testing?

No. Outsourced testing companies create smoke, regression, stress, and security tests, then execute those tests every time you ship them a release. They are testing your code against your specifications.

We are evaluating your specifications against the real world, and your software against the naivety of your users. Generally speaking, we are going to report any problem that would result in:

a potential customer not buying your software,
an existing customer not upgrading to your new version,
a customer calling you for support, and
a customer negatively referring you to a colleague.

For example, these are problems we might find that an outsourced testing company would not look for:

The system requirements for your software are not (or cannot be easily) found on your packaging (box, web site, etc.)
Your new software version increases your data file 500%
The icons in your web application are indistinguishable at an 8-bit color depth
Your application is vulnerable to SQL injection attacks

Isn't this just beta testing?

No. While we typically review software during its beta test phase, we are not testing the software like beta testers. Unlike the vast majority of beta testers, we:

follow a systematic and logical testing process
exercise your product on a variety of platforms and configurations
accurately categorize the severity of issues (e.g., data damaging, access preventing, annoying, or cosmetic)
recommend corrections that are based on accepted principles of UI design, human factor engineering, cognitive science, and information theory
provide reproduction and workaround instructions for technical problems we find
synthesize our findings into a cohesive evaluative report
answer any questions you have about our work

Are you suggesting your service is all I need to meet my testing needs?

No! Our service complements the other quality control methods you should already have in place: unit tests, hallway usability tests, peer review, regression and load testing, and beta tests to name a few.

Your goal is to write the best software possible, so that the most people possible will buy it. Our opinion alone will not make that happen. But, adding our service to your development plan will help eliminate problems before affecting your bottom line.

What if I find a problem in my software that you missed?

Please understand that we are not "testing" your application and in no way guarantee that we will find every problem in your program.

Our goal is to use your software like a real user would and then provide constructive feedback on our experience using language software developers understand. Our service in no way replaces unit, system, stress, or coverage testing that you should perform as part of your development.

However, if we missed something, we'd like to hear about it so that we can be sure to evaluate that area should you return to us for future evaluation services.

Can I redistribute the final report you produce?

Yes, just don't alter the contents or reprint without noting our authorship. Think of the report like a magazine article: copy it as much as you want, but please give us due credit.

Do you redistribute my final report?

No. We treat your report as completely confidential and will release it only to you.

We do reserve the right, however, to use snippets from your report for marketing on our web site. Should we use a snippet from your report, we will remove all identifying information to ensure your complete confidentiality.

Do you perform pro-bono reviews?

We do, on a case-by-case basis.

Open source offers a good example. We are huge proponents of open source software and, having written OSS ourselves, we know there's virtually no money in the game. One of our goals with this service is to help bring better software to the world at large, and we feel that OSS should be fairly reviewed even though there isn't any profit motivation.

So, don't hesitate to contact us and let us know the particulars of your situation. If we do perform a pro-bono review of your application, we ask that you mention our service in your application (e.g., on the about page or in program comments). We also appreciate a link back to our site from yours.

Should I use your service? I'm still not sure!

Regardless of whether you choose us, we strongly encourage you to get an independent, third-party review of your application. We would rather you choose a competitive service than choose to skip an independent evaluation.

Perhaps you'd like to check out Shareware Promotions' Software Evaluation Service. They offer similar services at a similar price.

Our Service

Our Guarantee