428 lines
17 KiB
TeX
428 lines
17 KiB
TeX
\chapter{Testing}
|
|
\section{Devices}
|
|
The following devices were used for the development and testing of MagSend. While theoretically MagSend should work on other Laptops and Smartphones I have only tested in on specified devices. For each device the most important specifications are listed.
|
|
|
|
// TOOD: Explain problem of testing over video calls
|
|
|
|
\subsection{Laptop}
|
|
The laptop was used as the sending device for MagSend. The Website was run using the Firefox browser and the laptop was always plugged in to the power grid to ensure that CPU doesn't throttle.
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{11.0cm}}
|
|
Device & Lenovo Thinkpad T470P \\
|
|
Operating System & Arch Linux \\
|
|
CPU & Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz \\
|
|
Core Count & 4\\
|
|
Memory & 16GB\\
|
|
Web-Browser & Firefox 108.0.1
|
|
\end{reqTblr}
|
|
\caption{Testing Device - Laptop\cite{ThinkpadT470p}}
|
|
\end{table}
|
|
|
|
\subsection{Smartphone}
|
|
The smartphone was used as the receiving device for MagSend. The app was run normally as a native Android application.
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{11.0cm}}
|
|
Device & Samsung Galaxy S9+ \\
|
|
Operating System & Android 10 with One UI 2.5 \\
|
|
SoC & Samsung Exynos 9810 \\
|
|
Memory & 6GB\\
|
|
Sensor Rate & Limited to 200 Hz by Android\cite{AndroidSensorLimit}
|
|
\end{reqTblr}
|
|
\caption{Testing Device - Smartphone\cite{GalaxyS9+}}
|
|
\end{table}
|
|
|
|
|
|
\section{Test Cases}
|
|
The following test cases are all structured in the same way. The table lists the related requirements from \autoref{chapter:requirements} and describes the test case briefly. Then the expected result row lists the steps and endresult that should happen for the test case to be successful. The last row describes the actual result from testing.
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T01 \\
|
|
Related Requirement & F01 \\
|
|
Description & The user is provided with the option the start calibration on the website. \\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the website in a web browser}
|
|
\item{The user is provided with a button to start the calibration}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
Upon opening the website, the user is greeted with an option to start the calibration (see \autoref{figure:t01}).
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 01}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t1.png}
|
|
\caption{Test Case 01 - Start Calibration Button on Website}
|
|
\label{figure:t01}
|
|
\end{figure}
|
|
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T02 \\
|
|
Related Requirement & F01 \\
|
|
Description & The user can start the calibration process on the website.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the website in a web browser}
|
|
\item{The user clicks on the "Start Calibration" button}
|
|
\item{The calibration process is started and the user is notified about it}
|
|
\item{During calibration the CPU is periodically stressed}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
When the user starts the calibration process the "Start Calibration" button changes to "Stop Calibration" and a message appears that indicates that calibration is in process (see \autoref{figure:t02}).
|
|
|
|
While calibrating the CPU is periodically stressed as seen in the CPU utilization graph (see \autoref{figure:t02_2}).
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 02}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t2.png}
|
|
\caption{Test Case 02 - Calibration in Process on Website}
|
|
\label{figure:t02}
|
|
\end{figure}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t2_2.png}
|
|
\caption{Test Case 02 - CPU Utilization}
|
|
\label{figure:t02_2}
|
|
\end{figure}
|
|
|
|
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T03 \\
|
|
Related Requirement & F01 \\
|
|
Description & The user can stop the calibration process on the website.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the website in a web browser}
|
|
\item{The user clicks on the "Start Calibration" button}
|
|
\item{The user then clicks on the new "Stop Calibration" button}
|
|
\item{The calibration is stopped and the CPU should not be utilized anymore}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
When the user clicks on the "Stop Calibration" button, the message about the calibration disappears and the button switches back to "Start Calibration" and the website should look like if it just has been opened.
|
|
|
|
After stopping the calibration, the CPU utilization also goes back down to normal levels (see \autoref{figure:t03_2}).
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 03}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t3_2.png}
|
|
\caption{Test Case 03 - CPU Utilization after Stopping the Calibration}
|
|
\label{figure:t03_2}
|
|
\end{figure}
|
|
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T04 \\
|
|
Related Requirement & F02 \\
|
|
Description & The user is provided with the option the start calibration in the app.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the app}
|
|
\item{The user is provided with a button to start the calibration}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
Upon opening the app, the user is greeted with an option to start the calibration (see autoref{figure:t04}).
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 04}
|
|
\end{table}
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T05 \\
|
|
Related Requirement & F02 \\
|
|
Description & The user can start the calibration process in the app.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the app}
|
|
\item{The user clicks on the "Calibrate" button}
|
|
\item{The user is shown the current measured magnetic field strength as well as a chart of the historical values}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
After starting the calibration in the app the user can see the magnetic field strength and historical data in a chart. (see autoref{figure:t05}).
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 05}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t5.jpg}
|
|
\caption{Test Case 05 - Calibrate Screen in the App}
|
|
\label{figure:t05}
|
|
\end{figure}
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T06 \\
|
|
Related Requirement & F02 \\
|
|
Description & The user can stop the calibration process in the app.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the app}
|
|
\item{The user clicks on the "Calibrate" button}
|
|
\item{While on the calibration screen the user clicks on the back arrow in the upper left corner}
|
|
\item{The user should be back on the home screen of the app}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
After clicking the back arrow inside the calibration screen the user arrives back on the home screen and can choose again between "Receive" and "Calibrate".
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 06}
|
|
\end{table}
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T07 \\
|
|
Related Requirement & F03 \\
|
|
Description & The user can enter text on the website.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the website in a web browser}
|
|
\item{The user clicks into the text input}
|
|
\item{The user can enter any text between 0 and 16 characters}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
The user can enter any character in the input he wants but he is not able to enter more than 16 characters (see \autoref{figure:t07}).
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 07}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\centering
|
|
\includegraphics[width=0.75\linewidth]{test_cases/t7.png}
|
|
\caption{Test Case 07 - Entering Text on the Website}
|
|
\label{figure:t07}
|
|
\end{figure}
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T08 \\
|
|
Related Requirement & F04 \\
|
|
Description & The entered text is validated on the website when sending.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the website in a web browser}
|
|
\item{The user clicks into the text input}
|
|
\item{The user enters any text between 0 and 16 characters}
|
|
\item{The user clicks on "Start Sending"}
|
|
\item{When the user hasn't entered any text or the text contains non-ASCII characters a fitting message is displayed and the sending process is not started.}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
When the user attempts to start sending when he hasn't entered any text, a message is displayed (see \autoref{figure:t08}).
|
|
|
|
When the user attempts to start sending when he has entered any non-ASCII characters, a message is diplayed. (see \autoref{figure:t08_2}).
|
|
|
|
In both cases the sending process is not started.
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 08}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t8.png}
|
|
\caption{Test Case 08 - Validation of Empty Input}
|
|
\label{figure:t08}
|
|
\end{figure}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t8_2.png}
|
|
\caption{Test Case 08 - Validation of Invalid Characters}
|
|
\label{figure:t08_2}
|
|
\end{figure}
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T09 \\
|
|
Related Requirement & F04 \\
|
|
Description & The user can send text using the website.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the website in a web browser}
|
|
\item{The user clicks into the text input}
|
|
\item{The user enters any text between 0 and 16 characters}
|
|
\item{The user clicks on "Start Sending"}
|
|
\item{The user is notified that the sending has started and the text that was entered is sent using the MagSend protocol by stressing the CPU according to the payload}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
When the user starts the sending process the "Start Sending" button changes to "Stop Sencing" and a message appears that indicates that the sending is is in process (see \autoref{figure:t09}).
|
|
|
|
In the CPU utilization during the sending, the structure of the preamble and the packet itself can be seen (see \autoref{figure:t09_2}).\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 09}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t9.png}
|
|
\caption{Test Case 09 - Website when Sending a Message}
|
|
\label{figure:t09}
|
|
\end{figure}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t9_2.png}
|
|
\caption{Test Case 09 - CPU Utilization when Sending a Message}
|
|
\label{figure:t09_2}
|
|
\end{figure}
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T10 \\
|
|
Related Requirement & F04 \\
|
|
Description & The user can stop the sending process.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the website in a web browser}
|
|
\item{The user clicks on the "Start Sending" button}
|
|
\item{The user then clicks on the new "Stop Sencding" button}
|
|
\item{The sending is stopped and the CPU should not be utilized anymore}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
When the user clicks on the "Stop Sending" button, the message about the sending disappears and the button switches back to "Start Sending" and the website should look like if it just has been opened.
|
|
|
|
After stopping the sending, the CPU utilization also goes back down to normal levels (see \autoref{figure:t10}).
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 10}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t10.png}
|
|
\caption{Test Case 10 - CPU Utilization after Stopping the Sending}
|
|
\label{figure:t10}
|
|
\end{figure}
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T11 \\
|
|
Related Requirement & F05 \\
|
|
Description & The user can receive data using the app.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the app}
|
|
\item{The user puts the smartphone on the laptop}
|
|
\item{The user clicks on the "Receive" button}
|
|
\item{The user starts the sending process on the website}
|
|
\item{The app will start the receive process and will indicate if the preamble was detected, the length of the received payload, and if the checksum is valid. The received text is also displayed.}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
When the users starts the receiving process he is informed about the current process of the receive status. He is informed whether the preamble was detected, the length of the payload and if the checksum is valid. He can also see the currently received text at all times (see \autoref{figure:t11}).
|
|
|
|
When the checksum is invalid the user is informed about it (see \autoref{figure:t11_2}).
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 11}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t11.jpg}
|
|
\caption{Test Case 11 - Receive Process without any Errors}
|
|
\label{figure:t11}
|
|
\end{figure}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t11_2.jpg}
|
|
\caption{Test Case 11 - Receive Process with Invalid Checksum}
|
|
\label{figure:t11_2}
|
|
\end{figure}
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T12 \\
|
|
Related Requirement & F05 \\
|
|
Description & The user can restart the receive process.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the app}
|
|
\item{The user puts the smartphone on the laptop}
|
|
\item{The user clicks on the "Receive" button in the app}
|
|
\item{The user starts the sending process on the website}
|
|
\item{The user clicks on the "Restart" button in the app}
|
|
\item{The app should reset the receive progress to a clean state}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
When the user clicks on the "Restart" button the received text gets removed and all status informations are reset to their default state.
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 12}
|
|
\end{table}
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T13 \\
|
|
Related Requirement & F05 \\
|
|
Description & The user can stop the receive process.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the app}
|
|
\item{The user puts the smartphone on the laptop}
|
|
\item{The user clicks on the "Receive" button in the app}
|
|
\item{While on the receive screen the user clicks on the back arrow in the upper left corner}
|
|
\item{The user should be back on the home screen of the app}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
After clicking the back arrow inside the receive screen the user arrives back on the home screen and can choose again between "Receive" and "Calibrate".
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 13}
|
|
\end{table}
|
|
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T14 \\
|
|
Related Requirement & N01 \\
|
|
Description & The app can receive data in airplane mode.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user enables airplane mode on his smartphone}
|
|
\item{The user opens the app}
|
|
\item{The user puts the smartphone on the laptop}
|
|
\item{The user clicks on the "Receive" button in the app}
|
|
\item{The app should still be able to receive messages over MagSend}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
Even in airplane mode the app can still receive data over MagSend as this process doesn't use any connectivity but only the inbuilt sensors of the smartphone (see \autoref{figure:t14}).
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 14}
|
|
\end{table}
|
|
|
|
\begin{figure}[H]
|
|
\includegraphics[width=0.5\linewidth]{test_cases/t14.jpg}
|
|
\caption{Test Case 14 - Receive Process in Airplane Mode}
|
|
\label{figure:t14}
|
|
\end{figure}
|
|
|
|
\begin{table}[H]
|
|
\begin{reqTblr}{l p{10.5cm}}
|
|
ID & T15 \\
|
|
Related Requirement & N02 \\
|
|
Description & The transfer speed between laptop and smartphone is \SI{1}{\bit\per\second}.\\
|
|
Expected Result &
|
|
\begin{enumerate}
|
|
\item{The user opens the app}
|
|
\item{The user puts the smartphone on the laptop}
|
|
\item{The user clicks on the "Receive" button}
|
|
\item{The user starts the sending process on the website}
|
|
\item{The transfer speed between laptop and smartphone should at least be \SI{1}{\bit\per\second}}
|
|
\end{enumerate} \\
|
|
Actual Result &
|
|
The clock speed of the transmitted signal is 500ms. As the MagSend payload is encoded using a manchester code, which doubles to data size, this means every second 1 bit gets transmitted. For more information about data rate benchmarks please refer to \autoref{chapter:benchmarking}.
|
|
\\
|
|
\end{reqTblr}
|
|
\caption{Test Case 15}
|
|
\end{table}
|