Beamr VISTA (VIsual Subjective Testing App) is a tool for collecting user opinions about visual quality for the purpose of development, calibration and verification of Beamr’s products. The app displays two versions of the same video side by side, and your task is to identify which version has lower quality. The app is not a test, and no score is given. The results will be averaged across a large set of users to form a subjective assessment of the video quality.
The VISTA app supports the following operating systems:
We recommend using at least a 2-core CPU with 4 GB of RAM.
The VISTA app opens in fullscreen, and you can use it on your internal laptop screen or an external computer screen. When using the app, you should be at a normal viewing distance from the screen, and try to avoid distracting factors such as light coming from behind the screen or projected directly on the screen.
You can download the software from the following links:
Double click on the downloaded ZIP file to unzip it, and after extraction click on the file named “vista” to launch the software.
When you launch the software, you will be asked to enter your email address or WorkerID. The email address is used only for correlating your results between different usages of the app. We recommend that you use your actual work or home email address to prevent conflict of emails with other users of the system. If you are using Amazon Mechanical Turk, enter your WorkerID.
After you enter your email address or WorkerID, the video files for the test will start loading from our server, and you will see a progress bar for the download. Once the download is finished, you will be presented with a short tutorial on using the app, and after the tutorial is complete you can start using the app.
When using the app, you will be shown two versions of the same video file, one on the right side of the screen and the other on the left side of the screen. Your task is to indicate which version has lower quality in your opinion, by pressing the buttons “Right has lower quality” or “Left has lower quality” which are displayed after the videos finish playback. There is no right and wrong answer, and no score will be given. You can repeat playback of the video pair by pressing the “Replay video” button. After the maximum number of video plays is reached, if you still can’t tell which video has lower quality after inspecting the videos, press one of the buttons anyway to move on to the next video pair. On the top of the screen you can see how many video files are left in the current session. When finished, the app will report the results to our servers and exit. If the application crashes or freezes, please upload the log as instructed and submit the task.
To reduce chances of the app crashing or hanging please wait few seconds after each video and before making your selection
Each time you run the VISTA app, a new set of videos will be downloaded. You can run the app as many times as you want, but we recommend to take a break of at least 15 minutes between runs to prevent eye strain which may affect the test results.