master_thesis/Chapters/Evaluation.tex

% !TeX root = ../Thesis.tex

%************************************************
\chapter{Evaluation}\label{ch:evaluation}
%************************************************
\glsresetall % Resets all acronyms to not used

% evaluation goal: use differential testing to analyze the security of esim on sim cards
% evaluation was performed on multiple esim on sim: in total 8 esims (shown in table)
% including the sysmoEUICC: special card -> loaded with gsma test certificate -> can also be used for smdp server with gsma test certificate -> testing of systems
% test ceritificate is freely available
% firmware implementation should not differ from normal esims
% altough for tests including an smdp server we have to use profiles with the test certificate
% estk.me does not offer an isd-r for regular interaction with our lpa implmenetation -> tesing is limited
% 9esim v2 offers both -> only testing isd-r not USAT for RSP related communication

The primary objective of this evaluation is to apply differential testing to analyze the behavior and potential security implications of various commercial eSIM on SIM solutions. This methodology aims to uncover inconsistencies, implementation deviations, and potential vulnerabilities by observing how different cards react to the same input sequences under controlled conditions.
To conduct a thorough and representative evaluation, we selected a diverse set of eSIM-on-SIM cards from multiple vendors. In total, \textbf{eight} different cards were included in the analysis, as shown in Table~\ref{tab:esim-overview}. These cards vary in terms of manufacturer, supported features, and firmware versions. The tests were conducted using the tracing, mutation, and fuzzing infrastructure described in \cref{ch:implementation}.

\todo{List Hardware specs}

\begin{table}[ht]
    \centering
    \begin{tabular}{|l|l|l|l|l|l|l|c|}
        \hline
        \textbf{Name} & \textbf{Manufacturer} & \textbf{Firmware} & \textbf{SVN} & \textbf{Apps} \\
        \hline
        \hline
        9esim v2     & Kigen            & 36.18.5 & 2.3.0 & 12 \\
        9esim        & Kigen            & 36.17.4 & 2.3.0 & 11 \\
        sysmoEUICC   & Kigen            & 35.6.35 & 2.3.0 & 8  \\
        EIOTClub     & Kigen            & 36.7.24 & 2.3.0 & 10 \\
        esim.me      & Eastcompeace     & 4.2.0   & 2.2.0 & 0  \\
        estk.me      & G+D              & 4.6.0   & 2.2.0 & 0  \\
        5Ber         & Eastcompeace     & 4.2.0   & 2.2.2 & 0  \\
        Xesim        & Eastcompeace     & 4.2.0   & 2.2.2 & 0  \\
        \hline
    \end{tabular}
    \caption{Overview of Analyzed eSIM Cards based on the information from \texttt{EuiccInfo2}.}
    \label{tab:esim-overview}
\end{table}

Among the tested cards, the \textbf{sysmoEUICC} is loaded with a \gls{gsma} test certificate, allowing it to interact with the official \gls{smdpp} test server infrastructure. This certificate is publicly available and enables the evaluation of remote profile management procedures in a realistic but controlled environment. The firmware implementation of the sysmoEUICC is assumed to be functionally equivalent to production-grade \glspl{esim}. However, this restricts use to use Profiles which are also signed by the \gls{gsma} test certificate.

In contrast, \textbf{estk.me} does not expose an \gls{isdr} for \gls{lpa}-based interaction, limiting the scope of evaluation for that platform. As such, only \gls{usat}-based interaction was possible, and no \gls{rsp}-related operations via the implemented testing framework could be performed. \todo{Check how estk.me implements rsp -> maybe rlpa etc}

The \textbf{9esim v2} card, in comparison, fully supports both \gls{isdr} access and \gls{usat} communication. Although \gls{usat} interfaces were excluded from evaluation for this particular card due to it not being supported by the testing framwork.

% general findings

% estk.me fwupd
% estk.me offers an esim firmware update utility on their website
% this tool can be run on some host machine and comes with the newest firmware image of the esim
% the firmware image seems to be encrypted when looking at the shannon entropy (run this through shannon entropy tool and add actual entropy)
% the firmware update tool is a binary
% analysation with ghidra -> rev engeneering
% looking at dissassembled code -> trying to guess functions based on their implementation
% this is something unique to the estk.me esim since all other esims neither offer an public update utility nor having a custom publicly documented flash endpoint for newer firmware
% Etsi (check which version) defines that all sims and esim have to have some sort of update utility
% application updates can be perfomred via global platform commands although requiring the manufacturer keys to unlock the smartcard


% update mechanism offers 2 functions: get_version and flash_firmware
% the custom flash endpoint is locate under the aid A06573746B6D65FFFFFFFF6677757064 for v1 and A06573746B6D65FFFFFFFFFFFF6D6774 for v2
% the card the i have for testing is an v1 version
% the aid is the hex encoded text 'estkmeÿÿÿÿfwupd' for v1 and 'estkmeÿÿÿÿÿÿmgt' for v2
% get_version retrieves the version code of the currently installed firmware by first selecting the aid and then sending the apdu packet APDU(class=0xAA, instruction=0xFF, p1=0x00, p2=0x00, le=0x08)
% our estk.me esim responds with Version 'T001V06' which corresponds to the 4.6.0 received via the euicc info 1
% this shows that this is an older estk.me model with the current one having the firmware version 'T3V7' as shown in newer firmware versions
% from the available firmware updates in the download page these versions exist: 
% T000V06-3.4.3
% T001V02-T001V05
% T001V03_V04-T001V05
% T001V05-T001V06
% T001V06-3.4.3
% T002V01-T002V06
% T002V06-3.4.3
% T003V03-4.2.4.1
% T003V03-4.4.3.1
% T003V03-4.5.1
% T003V03-T3V6SS0
% T3V4SS0-T3V6SS0
% T3V5SS0-T3V6SS0
% T3V6SS0-T3V7SS0 (v5.4.3)
% T001V06-3.4.3 seems to be the highest available version available for our card
% the list contains intermediate firmware versions which are used to upgrade from one major version to another i.e TXXXVXX-TXXVXX and current versions i.e TXXXVXX-X.X.X which are minor updates
% the list shows that there are 4 generations of cards: T000, T001, T002, and T003
% cards cannot be upgraded between generations
% only upugrade no downgrade support by the update mechanism -> autoselects the next firmware to update based on the installed one

% error handling is not really suported by the update mechanism -> wrong apdus or malformed apdus result in a CardConnectionException by the smartcard library which results from the pcsc-lite error SCARD_E_NOT_TRANSACTED (0x80100016)
% SCARD_E_NOT_TRANSACTED is thrown when the library tries to close a non existent connection \cite{https://pcsclite.apdu.fr/api/group__ErrorCodes.html} -> connection is already closed by the euicc -> euicc just closes the connection when erroring

% updating firmware
% the firmware update happens ins 6 Steps: setup, unlock, send_program_block, validate, and check_flash_status
% to initialise the fwup: select fwup endpoint and send APDU (cla=0x01, ins=0x55, p1=0x55, p2=0x55) -> init complete
% from here on the CLA for v0 APDUs is '00' and 'aa' for v1
% unlock the euicc: send apdu 'AA21000000'
% firmware is send in multiple blocks with the size 0x80D -> amount of blocks is defined by the firmware size i.e num_blocks = ceil(firmware_size / 0x80D)
% the mss is set to 255 bytes -> limit of normal APDU size defined by standard if not using extended APDUs (limit is 65536)
% each firmware block is sent in smaller APDUs where P1 indicates the current chunk of the program block and P2 the current_chunk & 0xFF
% after each program block is sent, the same data is sent again to euicc but the data part of the APDU is empty, the Le part of the APDU contains the chunk size and the instruction byte is set to 0x12 instead of 0x11 i.e go through each sent apdu and check that the received data length matches the chunk size provided in the Le of the APDU
% the update mechanism then sends the "AA13000000" APDU to check the flash status and indicate the next program block
% when all program blocks are sent -> send APDU 'AA00000000' to indicate the end of the flashing process
% using the information gathered through the reverse engeneering process -> reimplementation in python for further experimentation (also included in the resimulate as shown in implementation section TODO: cref to section)

% APDU mutation
% for APDU mutation we applied the same as shown in the mutation section i.e bitflip, random byte, zero block, shuffle block, truncate
% mutation are applied on a per chunk basis and in the same way when initially sending the program block
% same mutation is also applied when validating the blocks to ensure that the chunk size stays the same
% euicc accepts chunks initally until we send the check flash status APDU -> immediate crash
% indicates some sort of checksum or hash that is attached to each program block to ensure that the data received by the euicc is valid
% could not find any such thing in the binary -> road block thus could not find any further information about how this is beeing hashed or checksumed

% firmware binary analysis
% binary analysis was done with various tools like binwalk to check for known file types included in the firmware binary -> nothing
% entropy analysis with binwalk indicate that the firmware is encrypted (TODO: include image of entropy analysis)
% high shannon entropy of (TODO: check shannon entrpoy of all firmware versions)
% analysing the firmware files with ghidra also just showed just binary data without any structure

\section{estk.me Firmware}
\label{sec:estk_me_firmware}

Among all evaluated \glspl{esim}, \texttt{estk.me} stands out due to its publicly available firmware update utility, which is offered via their official website. This binary executable provides both the latest firmware images and the means to apply updates directly from a host machine, making it a unique case in comparison to other \gls{esim} vendors, none of whom publicly expose firmware updates or custom flashing endpoints. 

\subsubsection*{Firmware Structure and Analysis}

The firmware image accompanying the update utility appears to be encrypted or obfuscated. An entropy analysis conducted using the Shannon entropy metric indicates a consistently high entropy across all tested firmware files, suggesting the presence of encryption or compression. For instance, the entropy of the T001V06 firmware image was measured at approximately \texttt{7.998}, which is close to the theoretical maximum of \texttt{8.0} for purely random data.

\begin{figure}[htbp]
\centering
\begin{tikzpicture}
  \begin{axis}[
    title={Shannon Entropy},
    xlabel={Block},
    ylabel={Entropy},
    legend pos=south west,
    width=12cm,
    height=6cm,
    enlargelimits=true,
    mark size=2pt,
    xmin=0,
    ymin=0,
    grid,
  ]

    \addplot[
      color=blue,
      mark=*,
    ] table [x expr=\coordindex, y index=0] {Graphics/data/shannon_entropy_estk_me_t000v06_3_4_3.dat};
    \addlegendentry{T000V06}

    \addplot[
      color=red,
      mark=square*,
    ] table [x expr=\coordindex, y index=0] {Graphics/data/shannon_entropy_estk_me_t001v06_3_4_3.dat};
    \addlegendentry{T001V06}

    \addplot[
      color=green!60!black,
      mark=triangle*,
    ] table [x expr=\coordindex, y index=0] {Graphics/data/shannon_entropy_estk_me_t002v06_3_4_3.dat};
    \addlegendentry{T002V06}

  \end{axis}
\end{tikzpicture}
\caption{Shannon entropy values across blocks for three different firmware versions.}
\todo{Start with block 0, explain drop off at 200}
\end{figure}

A deeper static analysis using Ghidra~\cite{nsa_ghidra_2025} did not reveal any recognizable structure or file headers, further supporting the assumption of encryption. Similarly, tools like Binwalk~\cite{refirmlabs_binwalk_2025} did not detect known compression schemes, embedded file systems, or file signatures. Consequently, firmware payload analysis could not be meaningfully performed beyond block-level transmission.

\subsubsection*{Firmware Update Mechanism}

The update mechanism exposes two primary functions via a custom \gls{aid} endpoint:
\begin{itemize}
    \item \texttt{get\_version}: retrieves the currently installed firmware version.
    \item \texttt{flash\_firmware}: performs the actual firmware flashing process.
\end{itemize}

The \gls{aid}  used to access the update utility differs based on firmware generation. For example, the test card (generation T001) uses the \gls{aid}:
\begin{quote}
    \centering
    \texttt{A06573746B6D65FFFFFFFF6677757064} \\
    (hex-encoded: \texttt{'estkmeÿÿÿÿfwupd'})
\end{quote}
Firmware versions follow the format \texttt{TXXXVXX}, where major generation (\texttt{T000}--\texttt{T003}) and minor version are encoded. Firmware updates are incremental and strictly one-way, the tool automatically selects the next version based on the currently installed one, and downgrade paths are not supported.

\subsubsection*{Firmware Flashing Procedure}

The update process proceeds as follows:
\begin{enumerate}
    \item \textbf{Initialize}: Select the \gls{aid} and send the initialization \gls{apdu} \texttt{CLA=0x01, INS=0x55, P1=0x55, P2=0x55}.
    \item \textbf{Unlock}: Issue the unlock \gls{apdu} \texttt{AA21000000} to enable flashing.
    \item \textbf{Send Firmware Blocks}: 
        \begin{itemize}
            \item Firmware is divided into blocks of size \texttt{0x80D} bytes.
            \item Each block is sent in subchunks, using standard \glspl{apdu} (\texttt{CLA=0xAA}) with \texttt{INS=0x11}.
            \item After transmission, the same chunk is re-sent with \texttt{INS=0x12}, no payload, and the expected response length (\texttt{Le}) matching the original chunk size. This verification step is performed to ensure that the chunk was transmitted and received correctly, \ie, the \gls{euicc} confirms that the length of the previously received payload matches the expected length. We assume that the \gls{euicc} performs an internal consistency check between the declared length and the actual received bytes to validate chunk integrity.
        \end{itemize}
    \item \textbf{Check Flash Status}: The \gls{apdu} \texttt{AA13000000} is used to verify correct flash writing.
    \item \textbf{Finalize}: The \gls{apdu} \texttt{AA00000000} marks the end of the update process.
\end{enumerate}

The update tool fails gracefully only under specific conditions. For malformed or incorrect \glspl{apdu}, the \gls{euicc} abruptly terminates the connection, leading to a \texttt{CardConnectionException} raised by the PC/SC stack. This maps to the \texttt{SCARD\_E\_NOT\_TRANSACTED (0x80100016)} error, which occurs when the host attempts to communicate over a closed or non-existent connection~\cite{corcoran_pcsc-lite_2025}.

\subsubsection*{Reverse Engineering and Mutation Testing}

Using insights gained from disassembly in Ghidra, a Python-based reimplementation of the update mechanism was developed and available via the \gls{cli} as shown in \cref{sec:cli}. This allowed fine-grained control over the update process and enabled targeted mutation-based testing of firmware blocks.

The same mutation strategies as shown in \cref{subsubsec:mutation_engine}, such as bit flipping, byte shuffling, truncation, and others, were applied to the firmware blocks on a per-chunk basis during both initial transmission and block validation. Notably, mutated blocks were generally accepted until the \texttt{check\_flash\_status} step, where the \gls{euicc} would immediately terminate the connection. This behavior strongly suggests that the device internally verifies each block against a checksum, hash or cryptographic signature.



% tracing
% using simtrace 2 tracing and recording traffic showed that some esims use different aids for the isdr
% -> insert sample trace
% \begin{lstlisting}[style=logstyle, caption={Traced APDUs when launching the esim.me App}]
% INFO     Captured MANAGE CHANNEL  Apdu(00 70 00 00 01  01 9000)
% INFO     Captured SELECT MF/ADF.ISD-R Apdu(01 A4 04 00 10 a0000005591010ffffffff8900000100 6f1f8410a0000005591010ffffffff8900000100a5049f6501ffe0058203020200 9000)
% INFO     Captured MANAGE CHANNEL  Apdu(00 70 00 00 01  01 9000)
% WARNING  SELECT UNKNOWN AID a0000005591010000000008900000300
% INFO     Captured SELECT  Apdu(01 A4 04 00 10 a0000005591010000000008900000300  6a82)
% \end{lstlisting}
% line X shows that the selected aid could not be mapped to anything known
% probably done by vendors to force people to use their own lpa implementations eventough this is also limited by the ara-m
% we found the following isdr aids 
% xesim: A0000005591010FFFFFFFF8900000177
% 5ber: A0000005591010FFFFFFFF8900050500
% esim.me: A0000005591010000000008900000300
% esim.me offers both their custom aid and the common aid to select the isd-r

% replaying the traffic to other esims did not show any relevant or interisting behaviour
% limited to small subset of esim functionality due to usage of nonces for specific tasks

\section{Tracing}
\label{sec:eval_tracing}

To investigate vendor-specific behaviors in \gls{rsp}, we employed SIMTrace2 to capture the \glspl{apdu} exchanged between the \gls{lpa} running on a phone and the \gls{esim}. This enabled us to analyze the communication protocols used during profile management and \gls{euicc} interaction, especially focusing on the discovery and selection of the \gls{isdr}.

During the analysis of the \texttt{eSIM.me} eSIM, we observed the use of a custom \gls{aid} during the SELECT command for \gls{isdr}. The following listing illustrates a sample trace captured while launching the \texttt{esim.me} Android application:

\begin{lstlisting}[style=logstyle, escapeinside={(*}{*)}, caption={Traced APDUs when launching the esim.me App. Captured commands are executed by the esim.me App.}]
INFO     Captured MANAGE CHANNEL  Apdu(00 70 00 00 01  01 9000)
INFO     Captured SELECT MF/ADF.ISD-R Apdu(01 A4 04 00 10 a0000005591010ffffffff8900000100 6f1f8410a0000005591010ffffffff8900000100a5049f6501ffe0058203020200 9000)
INFO     Captured MANAGE CHANNEL  Apdu(00 70 00 00 01  01 9000)
WARNING  SELECT UNKNOWN AID a0000005591010000000008900000300(*\label{code:unkown_aid}*)
INFO     Captured SELECT  Apdu(01 A4 04 00 10 a0000005591010000000008900000300  6a82)
\end{lstlisting}

The log highlights a noteworthy behavior: in line \ref{code:unkown_aid}, the SELECT command targets an unknown \gls{aid} \texttt{a0000005591010000000008900000300}, which results in status word \texttt{6A82} (File not found). This custom \gls{aid} is not documented in standard \gls{gsma} or \gls{gp} specifications and appears to be vendor-specific.

This behavior suggests that certain vendors, such as \texttt{esim.me}, implement proprietary \glspl{aid} in parallel to the standardized \gls{isdr} \gls{aid} (\texttt{A0000005591010FFFFFFFF8900000100}). This may serve as an obfuscation or access control measure, potentially restricting third-party \gls{lpa} implementations or enforcing usage of vendor-specific software. However, due to the mandatory access mediation performed by the \gls{aram}, the effectiveness of such mechanisms is ultimately constrained by access control policies defined on the card.

We catalogued the \gls{isdr} \glspl{aid} observed during tracing across multiple \gls{esim} vendors:

\begin{itemize}
    \item \textbf{Xesim}: \texttt{A0000005591010FFFFFFFF8900000177}
    \item \textbf{5ber}: \texttt{A0000005591010FFFFFFFF8900050500}
    \item \textbf{eSIM.me}: \texttt{A0000005591010000000008900000300}
    \item \textbf{standard}: \texttt{a0000005591010ffffffff8900000100}
\end{itemize}

This variation in \gls{aid} usage may be driven by internal vendor policy, branding, or the need to segment access to multiple security domains or services. In the case of \texttt{eSIM.me}, the coexistence of both the standard and custom \glspl{aid} suggests a fallback mechanism for compatibility with generic \gls{lpa} implementations.

While tracing provides valuable insights into command sequencing and \gls{aid} selection behaviors, its utility is restricted when it comes to exercising full profile management flows or cross-device compatibility testing without access to valid cryptographic credentials.

% data fuzzing
% data fuzzing was performed on all esim except the estk.me esim since this card does not offer an isd-r 
% each fuzzing test was performed on all esims one after one

% findings data fuzzing
% data fuzzing succeeded for most cards
% ProfileInfoList failed for 9esim, 9esim v2 and EIOTCLUB
% Reported failure input was get_profiles(use_iccid=False, profile_class=None, tags=b'') with failure CardConnectionException from the smartcard library
% noticable was that the transaction LED on the CardReader continued to blink (suggesting APDUs were still being sent) eventough no traffic was generated by our code
% traffic LED continued to blink even when esim was removed
% only stopped blinking when completly disconnected from PC
% suggests possible bug in smartcard library

\section{Data Fuzzing}
\label{sec:data_fuzzing_evaluation}

\todo{Introduce different setups to make it more obvious when conducting seperate experiments} 


\begin{table}[h]
    \centering
    \caption{Data fuzzing results for 5ber, eSIM.me, and EIOTCLUB}
    \label{tab:data_fuzzing_result_part1}
    \begin{tabular}{lccc}
        \toprule
        \textbf{Function} & \textbf{5ber} & \textbf{eSIM.me} & \textbf{EIOTCLUB} \\
        \midrule
        SetDefaultDpAddress        & \cmark & \cmark & \cmark \\
        EuiccMemoryReset           & \cmark & \cmark & \cmark \\
        RetrieveNotificationsList  & \cmark & \cmark & \cmark \\
        ListNotification           & \cmark & \cmark & \cmark \\
        ProfileInfoList            & \cmark & \cmark & \xmark \\
        SetNickname                & \cmark & \cmark & \cmark \\
        PrepareDownload            & \cmark & \cmark & \cmark \\
        AuthenticateServer         & \cmark & \cmark & \cmark \\
        BoundProfilePackage        & \cmark & \cmark & \cmark \\
        \bottomrule
    \end{tabular}
\end{table}

\begin{table}[h]
    \centering
    \caption{Data fuzzing results for 9esim, 9esim v2, and Xesim}
    \label{tab:data_fuzzing_result_part2}
    \begin{tabular}{lccc}
        \toprule
        \textbf{Function} & \textbf{9esim} & \textbf{9esim v2} & \textbf{Xesim} \\
        \midrule
        SetDefaultDpAddress        & \cmark & \cmark & \cmark \\
        EuiccMemoryReset           & \cmark & \cmark & \cmark \\
        RetrieveNotificationsList  & \cmark & \cmark & \cmark \\
        ListNotification           & \cmark & \cmark & \cmark \\
        ProfileInfoList            & \xmark & \xmark & \cmark \\
        SetNickname                & \cmark & \cmark & \cmark \\
        PrepareDownload            & \cmark & \cmark & \cmark \\
        AuthenticateServer         & \cmark & \cmark & \cmark \\
        BoundProfilePackage        & \cmark & \cmark & \cmark \\
        \bottomrule
    \end{tabular}
    \todo{Merge tables and explain check mark and cross}
\end{table}

We conducted data fuzzing, as described in \cref{subsec:data_fuzzing}, on all tested \gls{esim} cards with the exception of \texttt{estk.me}. Each test case is executed sequentially across all eligible \glspl{esim} to ensure consistency and reproducibility of results.

The majority of the cards handled the fuzzed input data as expected, either processing the requests successfully or rejecting them gracefully with standard-compliant error responses. However, notable exceptions were observed during the execution of the \texttt{GetProfileInfo} test case as shown in \cref{tab:data_fuzzing_result_part1} and \cref{tab:data_fuzzing_result_part2}, particularly for the following cards:
\begin{itemize}
    \item 9esim
    \item 9esim v2
    \item EIOTCLUB 
\end{itemize}

In all three cases, the fuzzing input

\begin{lstlisting}[language=Python]
get_profiles(use_iccid=False, profile_class=None, tags=b'')
\end{lstlisting}

resulted in a \texttt{CardConnectionException} raised by the \texttt{smartcard} Python library during \gls{apdu} transmission. 

During these failures, a consistent and unusual hardware behavior was observed. The transaction LED on the card reader continued to blink, suggesting ongoing \gls{apdu} activity, even though no further commands were being issued by the fuzzing logic. This blinking persisted even after the test process was terminated and, in some cases, even after the \gls{esim} card was physically removed from the reader.

The LED activity ceased only when the card reader was fully disconnected from the host machine. This behavior strongly indicates that the failure triggered an inconsistent or undefined state within the underlying \texttt{smartcard} library or \texttt{libpcsc}.

% Although this failure was not directly traceable to a specific eSIM firmware implementation (due to the exception occurring before a meaningful response could be recorded), its repeatability across multiple cards and hardware sessions suggests it warrants further investigation—potentially outside the scope of this work but relevant for tooling robustness in future studies.


% apdu fuzzing

% optimizing for coverage
% problem: aggressiv mutating in the beginning prevents gaining coverage when succeeding in a mutation, mutation succeeded but failed immediatly afterwards due to more mutations beeing applied
% solution: as soon as mutation succeed do not use any more mutation in the initial run directly after the successful mutation i.e. first try to finish the run the test out further mutations
% experiments were conducted with a HID OMNIKEY 3121 USB smart card reader
% esims were inserted into the sysmocom SIM card adapter to adapt the 2FF eSIM to full-size
% adapter was inserted into smart card reader
% profile used for regular consumer cards: "LPA:1$rsp.truphone.com$QR-G-5C-1LS-1W1Z9P7"
% profile used with the sysmoeuicc due to this having the GSMA test certificate installed: "LPA:1$smdpp.test.rsp.sysmocom.de$TS48v5_SAIP2.1B_NoBERTLV"

% differences in execution time
% depends on chip speed and how likely the esim accepts mutated apdus -> more branches -> more apdus
% 

% Encountered Errors

% SCP03TSecurityError
% refers to an error when working with the Secure Channel Protocol 3 -> security protocol
% suffix T in SCP03T refers to Terminal side of application as opposed to SCP03C (Card) or SCP03S (Server)
% cause can have many origins but mostly happens during session establishment or secure messaging
% in our case its thrown during the LoadBoundProfile step specificially when sending sequenceOf86, sequenceOf88, and firstSequenceOf87

% ApduException
% caused by malformed asn1 structure

% InvalidCertificate
% occured during execution of AuthenticateServer and PrepareDownload 
% certificate validation failed

% InvalidSignature
% initialiseSecureChannelRequest was the only function that caused this during
% signature verification to establish secure channel failed during initialiseSecureChannelRequest

% UnsupportedRemoteOpType
% only thrown during initialiseSecureChannelRequest
% is always set to installBoundProfilePackage (1)
% mutation ZERO_BLOCK sets the value to zero and truncate removes it entirely

% UnsupportedCurve
% bitflip caused certificate curve value to be not supported by the euicc
% InvalidTransactionID

% UndefinedError
% thrown during the authenticate_server and firstSequenceOf87 function
% indicates that the euicc does not know how to handle this error

% DecodeTagError
% thrown by the python asn1tools parser -> happend on lpa side 
% function was authenticate_server
% Complete Error message: AuthenticateServerResponse.authenticateResponseOk.euiccSignature1: Expected OCTET STRING(euiccSignature1) with tag '5f37', but got '2474'. (At offset: 271)
% indicates that euiccSigned1 is shorter then indicated by the ber encoding tag
% 5f37 appears at offset 261
% adjusting the specific tags to make it decodable shows that there is now differenence to a successfully decoded AuthenticateServerResponse apdu apart from the euiccSignature1, serverChallenge and transactionId. All of which are supposed to be different 
% still interisting that the response data was malformed


\section{APDU Fuzzing}
\label{sec:apdu_fuzzing}

To evaluate the robustness of \gls{rsp} protocol handling and smart card behavior under malformed input, we applied mutation-based fuzzing at the \gls{apdu} level. The goal was to explore unintended code paths and observe crash conditions or unexpected responses.

\subsection*{Optimizing for Coverage}
Initial fuzzing experiments revealed that applying aggressive mutations across all \glspl{apdu} early in a transaction significantly hindered code path exploration. In many cases, although a mutation in an early \gls{apdu} succeeded in provoking an altered behavior or state change, subsequent mutated \glspl{apdu} caused premature failures. As a mitigation strategy, we adopted a greedy mutation approach: once a mutation led to a successful state transition, the subsequent \glspl{apdu} in that session were executed unmutated to allow complete transaction processing and thereby maximize coverage.

\subsection*{Experimental Setup}
All tests were conducted using a HID OMNIKEY 3121 USB smart card reader.\glspl{esim} were inserted into a Sysmocom 2FF-to-full-size adapter to ensure compatibility with the reader. For consumer-grade \glspl{esim}, the following profile was used:
\todo{Reference where the profiles are comming from}
\begin{center}
\texttt{LPA:1\$rsp.truphone.com\$QR-G-5C-1LS-1W1Z9P7}
\end{center}

For the \texttt{sysmoEUICC}, which uses the \gls{gsma} test certificate, the following test profile was employed:
\begin{center}
\texttt{LPA:1\$smdpp.test.rsp.sysmocom.de\$TS48v5\_SAIP2.1B\_NoBERTLV}
\end{center}

The execution time of fuzzed \gls{apdu} sequences varied depending on chip processing speed and the degree to which the mutated input triggered deeper protocol branches.

\subsection*{Observed Errors}
The following classes of errors were consistently encountered during mutation campaigns:

\todo{Put errors into a table with example mutations and explain in text}
\begin{itemize}
    \item \textbf{SCP03TSecurityError}: Occurred during the \texttt{\justify LoadBoundProfilePackage} step, particularly when transmitting \texttt{sequenceOf86}, \texttt{sequenceOf88}, or the initial \texttt{sequenceOf87}. This indicates failure during Secure Channel Protocol 03 (terminal-side variant) session establishment.
    
    \item \textbf{ApduException}: Triggered by malformed \gls{asn1} structures, typically due to mutations altering length or tag fields.

    \item \textbf{InvalidCertificate}: Observed during \texttt{AuthenticateServer} and \texttt{PrepareDownload}. The \gls{euicc} rejected the server certificate during validation.

    \item \textbf{InvalidSignature}: Raised exclusively during \texttt{\justify InitialiseSecureChannelRequest}, indicating that the \gls{euicc} failed to verify the signature required for secure channel establishment.

    \item \textbf{UnsupportedRemoteOpType}: Also restricted to \texttt{\justify InitialiseSecureChannelRequest}. Mutation operators such as \texttt{ZERO\_BLOCK} or \texttt{TRUNCATE} corrupted the operation type field, which is normally set to \texttt{\justify installBoundProfilePackage (1)}.

    \item \textbf{UnsupportedCurve}: Introduced via bit-level mutations affecting certificate parameters. The \gls{euicc} did not support the altered elliptic curve definition.

    \item \textbf{UndefinedError}: A non-specific error raised by the \gls{euicc} during \texttt{AuthenticateServer} and \texttt{FirstSequenceOf87}, implying the input could not be classified under known error types.

    \item \textbf{DecodeTagError}: Thrown by the Python-side \gls{asn1} decoder during parsing of malformed responses. A representative example:
    \begin{quote}
        \texttt{euiccSignature1: Expected OCTET STRING (tag '5f37'), but got '2474'. (At offset: 271)}
    \end{quote}
    This indicates an inconsistency between the \gls{ber}-encoded tag and its expected value, likely due to mutated length fields or premature truncation. Notably, this malformed \gls{apdu} was still accepted and responded to by the \gls{euicc}, albeit with corrupt data.

\end{itemize}


% successful mutations
% analyzing recorded apdu fuzzing
% PATHS = glob.glob("recordings/*.resim")
% 
% for path in PATHS:
%     with open(path, "rb") as f:
%         try:
%             recorder: OperationRecorder = pickle.load(f)
%         except Exception as e:
%             print(f"Failed to load {path}: {e}")
%             continue
% 
%         print(f"Processing {path}")
% 
%     def get_successfully_mutated_data(tree: MutationTreeNode) -> bool:
%         if tree.leaf:
%             return tree.failure_reason is None
% 
%         has_successful_child = False
% 
%         for child in tree.children:
%             is_successful = get_successfully_mutated_data(child)
% 
%             if is_successful:
%                 has_successful_child = True
% 
%             if (
%                 is_successful
%                 and tree.recording
%                 and tree.mutation_type != MutationType.NONE
%             ):
%                 diff = difflib.ndiff(
%                     tree.recording.original_apdu.data.hex(),
%                     tree.recording.mutated_apdu.data.hex(),
%                 )
%                 pprint(
%                     {
%                         "recording": path,
%                         "function": tree.func_name,
%                         "mutation_type": tree.mutation_type,
%                         "original_apdu": tree.recording.original_apdu.data.hex(),
%                         "mutated_apdu": tree.recording.mutated_apdu.data.hex(),
%                         "diff": "".join(diff),
%                     }
%                 )
% 
%         return has_successful_child
% 
%     get_successfully_mutated_data(recorder.root)
% 
% load recordings
% loop through nodes and find nodes where the branch is successfull and the mutation type is not none
% shows all successful mutations in the recodings folder
% ndiff: compares two strings and returns a diff view that shows the differences for both strings
% 

% get_euicc_info_1 truncate success
% truncation of trailing 00
% bf2000 -> bf20

% firstSequenceOf87 bitflip success
% diff:  a- 0+ 1  1  a  8  7  1  8  8  b  0  c  9  1  6  5  4  1  1  3  9  '
%         'd  f  b  1  d  4  f  0  6  3  c  7  b  c  2  d  7  4  b  d  f  1  2  '
%         'e  b  0  7  8  0  4  5  e  0  0  b'
% a0 -> a1
% indicates another channel was used

% authenticate_server was truncated but still successful
% last 75% of authenticate_server apdu are truncated
% significant portion of the authenticate server request is missing -> 210 bytes missing
% -> investigate what is missing 
% trying to decode the mutated apdu leads to failure due to wrong length indicators in asn1 encoding -> manually fixing them
% missing parts in AuthenticateServerRequest: ctxParams1; signature, signature algorithm of serverCertificate, certificate extensions (2.5.29.14, 2.5.29.17, 2.5.29.35)

% further analyzation
% specs: root of trust is always GSMA -> trust anchor, chain validation is important
% "The server certificate shall be verified against the Root of Trust of the eUICC, and the digital signature of the AuthenticateServerRequest shall be verified." - SGP.22
% certificate is CERT.DPauth.ECDSA (SGP.22 naming)
% bug cannot be triggered directly by truncating the AuthenticateServerRequest -> only able to be trigger when performing other failed mutation requests beforehand
% to trigger bug: 
% - perform one authenticate_server request with for example a bitflip -> authenticate_server request fails
% - perform full profile_provisioning with truncated server certificate
% when triggerign directly: euicc returns UndefinedError error
% first theory: has to be some issue in the euicc state machine for the rsp process

% only happening on Eastcompeace euiccs

% signature of the certificate signs the whole tbsCertificate -> signature apparently not checkt and validated
% 

% "The Server (the entity providing the function, e.g., SM-DP+) SHALL be authenticated first by the Client (the entity requesting the function). Authentication SHALL include the verification of a Server Certificate chain ending at an eSIM CA RootCA Certificate (section 4.5.2)." from RSP
% test if we can exploit this: provision a valid profile from my own non GSMA accredited smdp+ server using the osmo-smdpp server included in pysim
% we modify the server in way that it ignores checks that its own hostname is the subject of the CERT.DPauth.ECDSA Certificate
% we run the initial AuthenticateServerRequest using a valid profile and applying mutation that fails to prepare the esim
% before sending the second AuthenticateServerRequest, we switch out the subjectPublicKeyInfo for our own controlled publicKey
% as explained in section (cref to background section) the euicc uses the server provided certificates public key to perform the ECDH Key agreement 
% this results in a shared session key that is used for encrypting and authenticating further communication
% esim happily accepts the public provided by use -> does not check if signature is still valid nor checks the certificate chain if the certificate is signed by GSMA root CA  

% doing the same experiment but using our own profile to perform the initial authenticateServerRequest -> check if the esim uses the certificate from the first failed authentication for further communication
% 1. Perform failed mutation with our own profile and smdpp server
% 2. Use valid profile with valid smdpp server but use truncation mutation to cutoff the signature and some extensions
% results in an unkown error -> suggests that the euicc uses the certificate from the first failed authenticateServerRequest

% persistency accross power cycles
% steps similar to previous test
% 1. provision valid profile from valid smdpp with bitflip -> failed authenticateServerRequest
% 2. remove esim from adapter card, wait 5 seconds and reinsert
% 3. provision valid profile from valid smdpp with truncation -> signature is removed -> successful installation
% Cross-Session certificate resuse is still possible after power cycle of the esim

% check if possible when using different valid profiles on same smdpp
% use "LPA:1$rsp.truphone.com$QR-G-5C-1LS-1W1Z9P7" activation code for initial provisioning with bitflip
% use "LPA:1$rsp.truphone.com$QRF-SPEEDTEST"  activation code for truncated provisioning
% successfull mutual authentication aswell as profile binding and download

% using different smdpp
% 1. 'LPA:1$rsp-eu.redteamobile.com$5901981126831169'
% 2. 'LPA:1$rsp.truphone.com$QRF-SPEEDTEST'
% results in UndefinedError during the authenticateServerRequest -> probably due to not being able to decode the server signed information of the new request with the public key of the previously sent Certificate

% based on this information it would suggest that when we sent the first request but switch the public key to the one from the second request -> would leed to successful installation
% still results in UndefinedError Exception
% same result when switching out the whole certificate
% suggests that more than just the certificate is beeing reused from the previouse session

% looking closer into the first bitflip in the failed request when using different smdps
% first flip is in the tag for the AuthenticateServerRequest (bf38 -> be38 where be38 is invalid) -> fixing this leads to the following other flips
% flips are done in euiccPkiToBeUsed, serverCertificate.serialNumber, serverSignature1, serverSigned1.euiccChallenge, serverSigned1.serverChallenge
% when looking at the bitflips from the failed request with the same smdp server
% first flip again in the tag for the request, fixing this shows flips in euiccCiPKIdToBeUsed, serverCertificate.issuer, serverCertificate.serialNumber, serverSignature1, serverSigned1.serverChallenge


\subsection{Analysis of Successful Mutations}
\label{subsec:successful_mutations}

To further understand the behavior of different \glspl{euicc} \todo{Which euiccs?} under mutation-based fuzzing, we post-processed all recorded \gls{apdu} traces.

\subsubsection*{Mutation Examples}

\paragraph{GET\_EUICC\_INFO\_1 Truncation}
A truncated \gls{apdu} with trailing padding bytes removed (e.g., \texttt{BF2000} $\rightarrow$ \texttt{BF20}) was still accepted and processed successfully by several \glspl{euicc}. This indicates that, for some non-critical operations, strict \gls{ber}/\gls{der} tag integrity is not enforced.

\paragraph{FIRSTSEQUENCEOF87 Bitflip}
A single-bit mutation changed a byte from \texttt{0xA0} to \texttt{0xA1}. Despite the low-level alteration, the \gls{apdu} was accepted, suggesting a fallback to an alternate logical channel or slightly different interpretation of \gls{tlv} structure.

\paragraph{AUTHENTICATE\_SERVER Truncation}
In one case, the mutation engine truncated approximately 75\% of the \texttt{AuthenticateServerRequest} \gls{apdu}—removing critical fields such as \texttt{ctxParams1}, the digital signature, and certificate extensions (including \texttt{2.5.29.14}, \texttt{2.5.29.17}, \texttt{2.5.29.35}). The \gls{apdu} failed \gls{asn1} decoding due to inconsistent length indicators; however, after manual correction, analysis revealed that the missing fields did not prevent the \gls{euicc} from accepting the request under specific conditions.

\todo{Try to find some more fuzzing statistics i.e runs, mutations per second, explored paths}

\subsection{Certificate Validation Bypass}
\label{subsec:certificate_bypass}

Further experimentation revealed a state machine vulnerability in the \gls{rsp} implementation of \glspl{euicc} manufactured by Eastcompeace. This vulnerability allows partial bypass of the \gls{gsma} certificate chain validation under specific mutated sequences of \texttt{AuthenticateServerRequest} messages.

\subsubsection*{Expected Behavior}
According to SGP.22:
\begin{quote}
``The Server (the entity providing the function, e.g., \gls{smdpp}) SHALL be authenticated first by the Client (the entity requesting the function). Authentication SHALL include the verification of a Server Certificate chain ending at an \gls{esim} \gls{ca} RootCA Certificate (section 4.5.2).''~\cite{gsma_sgp22_2025}
\end{quote}
The server certificate $Cert_{Sa}$ must be verified against the \gls{gsma} root-of-trust $Cert_{CI}$, and the digital signature of the \texttt{AuthenticateServerRequest} must be valid.

\subsubsection*{Observed Behavior}
By combining a series of failed and mutated authentication requests, it was possible to trigger incorrect trust decisions by the \gls{euicc}:

\begin{enumerate}
    \item A first \texttt{AuthenticateServerRequest} containing a malformed or mutated certificate (e.g., bit-flipped data) is sent to the \gls{euicc}. As expected, this request fails.
    
    \item A second \texttt{AuthenticateServerRequest} is sent, using a valid profile but with the certificate $Cert_{Sa}$ truncated (i.e., digital signature and extensions removed). Surprisingly, the \gls{euicc} accepts this message and continues with the profile installation process.

    \item \gls{asn1} decoding of the truncated certificate $Cert_{Sa}$ on the \gls{lpa} side fails due to missing signature fields, but the \gls{euicc} does not reject it, indicating that signature verification was skipped.

    \item Swapping the \texttt{SubjectPublicKeyInfo} in the truncated certificate $Cert_{Sa}$ with an attacker-controlled public key $SK_A$ still results in successful mutual authentication and secure channel establishment. This would imply that the \gls{euicc} did not use the attacker controlled public key, but instead reused the previously sent publlic key from the failed request, as otherwise the \gls{smdpp} would have not been able to verify the signature in the subsequent client authentication $\mathrm{Cert}_{Sa} \triangleleft \mathrm{Cert}_{CI}$ as shown in \cref{fig:authenticate_server_sd}.
\end{enumerate}

\begin{figure}[h!]
	\centering
    \includesvg[width=\textwidth]{Graphics/authenticate_server_sd.svg}
    \caption{Sequence diagram of the authenticate server process.\cite{ahmed_security_2024}}
    \label{fig:authenticate_server_sd}
\end{figure}

A deeper analysis of mutated \gls{apdu} payloads of the initial \texttt{AuthenticateServerRequest}, particularly in cases using different \gls{smdpp} servers, shows the following:

\begin{itemize}
    \item The first mutation typically occurs in the tag of the \texttt{AuthenticateServerRequest}, where the correct \texttt{BF38} tag is flipped to an invalid \texttt{BE38}. Correcting this tag provides a valid \gls{asn1} structure and makes the remaining data decodable.
    
    \item In cross-server scenarios (different \gls{smdpp}), subsequent flips occur in fields such as \texttt{euiccPki\-ToBeUsed}, \texttt{server\-Certificate.\-serial\-Number}, \texttt{server\-Signature1}, and both the \texttt{euicc\-Challenge} and \texttt{server\-Challenge} components of \texttt{server\-Signed1}. This attempt will not be able to pass the bug.
    
    \item In same-server scenarios (same \gls{smdpp}), a similar flip in the tag is observed. Upon correction, mutations manifest in \texttt{euiccCiPKIdToBeUsed}, \texttt{serverCertificate.issuer}, \texttt{serverCertificate.serialNumber}, \texttt{serverSignature1}, and again in \texttt{serverSigned1.serverChallenge}. This scenario will be able to pass the truncation bug an successfully install the profile.
\end{itemize}

These observations further validate the theory that the certificate from the initial failed \texttt{AuthenticateServerRequest} is cached and reused by the \gls{euicc}. In both the same-server and cross-server mutation scenarios, the public key remains unmodified, yet the behavior diverges based on the prior session’s state. Additionally, certificate validation $\mathrm{Cert}_{Sa} \triangleleft \mathrm{Cert}_{CI}$ appears to be bypassed entirely, as the mutated certificate $Cert_{Sa}$ contents never represent a structurally valid or correctly signed certificate at any point in the provisioning flow.

\subsubsection*{State Persistence Across Sessions}
Further experiments showed that this vulnerability persists across sessions and even power cycles:

\begin{itemize}
    \item A failed mutated authentication is followed by a successful truncated certificate installation—even after the \gls{euicc} is physically removed and reinserted into the card reader.
    
    \item If different profiles from the same \gls{smdpp} are used (e.g., activation codes \texttt{QR-G-5C-1LS-1W1Z9P7} and \texttt{QRF-SPEEDTEST}), the bypass remains possible.
    
    \item However, if different \gls{smdpp} servers are used (e.g., \texttt{rsp.truphone.com} vs \texttt{rsp-eu.redteamobile.com}), the attack fails, and the \gls{euicc} returns an \texttt{UndefinedError}. This supports the idea that the \gls{euicc} reuses the public key from the previously sent but failed \texttt{AuthenticateServerRequest}.
\end{itemize}

To further probe the caching mechanism, additional tests were conducted in which the public key of the second \texttt{AuthenticateServerRequest} was substituted into the first request. Based on previous observations, it was hypothesized that this would allow a successful installation by aligning the reused state with the new request.

However, these attempts consistently resulted in an \texttt{UndefinedError} exception. The same error occurred even when replacing the entire certificate with the one used in the second request. This strongly suggests that the \gls{euicc} caches more than just the certificate object, potentially including intermediate cryptographic material or session-specific internal state derived from the original malformed request.

This reinforces the assumption that the \gls{euicc} maintains non-trivial persistent state across sessions, and that this state influences the acceptance of subsequent messages even when they originate from clean profiles or well-formed certificates.

% \todo{Find out which parts are reused aswell -> serverSigned1, serverSignature}
% \todo{serverSigned1 -> check if signature is verified}

\subsubsection*{Implications}
The observed behavior violates \gls{gsma} security requirements, particularly in Sections 4.5.1 and 4.5.2 of SGP.22~\cite{gsma_sgp22_2025}, which mandate certificate chain validation and signature verification during server authentication. The following points summarize the core issues:

\begin{itemize}
    \item The \gls{euicc} reuses elements (possibly the previously received certificate or derived session keys) from failed \texttt{AuthenticateServerRequest} sessions.
    \item Certificate truncation alone does not prevent the \gls{euicc} from proceeding with secure channel establishment.
    \item The signature over the \texttt{tbsCertificate} is not validated after session caching has occurred.
\end{itemize}