89 lines
4.5 KiB
TeX
89 lines
4.5 KiB
TeX
\chapter{Benchmarking}
|
|
\label{chapter:benchmarking}
|
|
This chapter will research the real world capabilities of MagSend. I attempt to transmit a message using different speeds of MagSend under different system CPU loads to see how it performs and which transfer speed is the most reliable to use.
|
|
|
|
The following benchmark test are structured in the same way. I run each test 5 times to get a more accurate view of the test. For each test I note if MagSend can detect the preamble, the received payload length, whether the \gls{crc} is valid and the received message.
|
|
|
|
The message used in the test transmissions is "\textbf{Test}".
|
|
|
|
|
|
\section{Idle CPU Load}
|
|
The following benchmarks were run during idle CPU Load, which was about 1-5\% utilization.
|
|
|
|
\subsection{\SI{0.5}{\bit\per\second}}
|
|
\begin{table}[H]
|
|
\begin{bfhTabular}{l l l l l}
|
|
Run & Preamble Detected & Payload Length & Is CRC Valid & Received Message \\
|
|
1 & Yes & 4 bytes & No & Vêÿ\\
|
|
2 & Yes & 4 bytes & Yes & Test \\
|
|
3 & Yes & 4 bytes & Yes & Test\\
|
|
4 & Yes & 4 bytes & Yes & Test \\
|
|
5 & Yes & 4 bytes & No & {\textunderscore}rÛ
|
|
\end{bfhTabular}
|
|
\caption{Benchmark - Idle CPU Load - \SI{0.5}{\bit\per\second}}
|
|
\end{table}
|
|
|
|
\subsection{\SI{1}{\bit\per\second}}
|
|
\begin{table}[H]
|
|
\begin{bfhTabular}{l l l l l}
|
|
Run & Preamble Detected & Payload Length & Is CRC Valid & Received Message \\
|
|
1 & Yes & 4 bytes & Yes & Test\\
|
|
2 & Yes & 4 & No & Test\\
|
|
3 & Yes & 4 & Yes & Test\\
|
|
4 & Yes & 4 & Yes & Test\\
|
|
5 & Yes & 4 & No & TÊæé
|
|
\end{bfhTabular}
|
|
\caption{Benchmark - Idle CPU Load - \SI{1}{\bit\per\second}}
|
|
\end{table}
|
|
|
|
\section{25\% CPU Load}
|
|
The following benchmarks were run with 25\% CPU utilization. The tool \texttt{stress} was used to put a load on the CPU \cite{stress}.
|
|
|
|
\subsection{\SI{0.5}{\bit\per\second}}
|
|
\begin{table}[H]
|
|
\begin{bfhTabular}{l l l l l}
|
|
Run & Preamble Detected & Payload Length & Is CRC Valid & Received Message \\
|
|
1 & Yes & - & - & -\\
|
|
2 & Yes & \SI{4}{\byte} & - & -\\
|
|
3 & No & - & - & -\\
|
|
4 & No & - & - & -\\
|
|
5 & No & - & - & -\\
|
|
\end{bfhTabular}
|
|
\caption{Benchmark - 25\% CPU Load - \SI{0.5}{\bit\per\second}}
|
|
\end{table}
|
|
|
|
\subsection{\SI{1}{\bit\per\second}}
|
|
\begin{table}[H]
|
|
\begin{bfhTabular}{l l l l l}
|
|
Run & Preamble Detected & Payload Length & Is CRC Valid & Received Message \\
|
|
1 & No & - & - & -\\
|
|
2 & Yes & - & - & -\\
|
|
3 & No & - & - & -\\
|
|
4 & No & - & - & -\\
|
|
5 & No & - & - & -
|
|
\end{bfhTabular}
|
|
\caption{Benchmark - 25\% CPU Load - \SI{1}{\bit\per\second}}
|
|
\end{table}
|
|
|
|
\subsection{Results}
|
|
As one can see in the benchmarks, the most successful configuration was running MagSend with \SI{1}{\bit\per\second} at with an idle CPU followed by \SI{0.5}{\bit\per\second} at an idle CPU.
|
|
|
|
The benchmarks with 25\% CPU load were mostly failing, with some few ones being able to detect the preamble and one even being able to receive the payload length from the message. There might be a few reasons why the results are like this. The first one is thermal throttling. CPUs produce a lot of heat when under load and thus they get hotter. When they get to hot, they throttle by reducing clock speeds. This process is designed to protect the CPU from overheating \cite{thermalThrottling}.
|
|
|
|
The laptop that was used for running the benchmarks, already reaches CPU temperatures of over \SI{90}{\celsius} when only running at around 25\% CPU load (see \autoref{figure:cpu_temperatures}).
|
|
|
|
\begin{figure}[H]
|
|
\fbox{\includegraphics[width=1\textwidth]{benchmarking/cpu_temperatures.png}}
|
|
\caption{CPU temperatures under 25\% CPU load}
|
|
\label{figure:cpu_temperatures}
|
|
\end{figure}
|
|
|
|
When MagSend then attempts to sent data by putting even more stress on the CPU, the CPU begins to throttle and the resulting CPU loads gets dampened (see \autoref{figure:throttle}). While longer spikes like the preamble at the start of the packet are still visible, the rest of the message is only noise and really be received.
|
|
|
|
\begin{figure}[H]
|
|
\fbox{\includegraphics[width=1\textwidth]{benchmarking/1bps_25.png}}
|
|
\caption{CPU temperatures under 25\% CPU load}
|
|
\label{figure:throttle}
|
|
\end{figure}
|
|
|
|
The other problem with the failing benchmarks is on the receiving side. The interval at which the sensor provides data to the application is not guaranteed and thus introduces some jitter \cite{androidSensorMonitor}.
|