diff --git a/source/Lab01/Guide/assignment.tex b/source/Lab01/Guide/assignment.tex index 2025d2d..45c9500 100644 --- a/source/Lab01/Guide/assignment.tex +++ b/source/Lab01/Guide/assignment.tex @@ -2,13 +2,13 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} \usepackage[utf8x]{inputenc} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% SELECT ONE OF THE FOLLOWING PACKAGES FOR THE LANGUAGE +% SELECT ONE OF THE FOLLOWING PACKAGES FOR THE LANGUAGE \usepackage[english]{babel} % \usepackage[portuges]{babel} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -17,8 +17,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -68,16 +68,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 1 \\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & 14 Sept 2020 \\ +\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Background: Distributed Systems} & \textbf{Due Date:} & 21 Sept 2020\\ +\multicolumn{2}{|l|}{Background: Distributed Systems} & \textbf{Due Date:} & \\ \hline \end{tabularx}} \rhead{} @@ -92,7 +92,7 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 \section*{Preliminary Notes} -This class recalls background on distributed systems and crpytography, essencial building blocks for studying blockchain technology. This laboratory is based on several sources \cite{dscd,sdist2017,Wong2014,Ousterhout_presentation,Verissimo2001,correia2019byzantine,raft_paper}. We recommend students to conduct additional research on other sources \cite{mit,princeton}. +This class recalls background on distributed systems and cryptography, essential building blocks for studying blockchain technology. This laboratory is based on several sources \cite{dscd,sdist2017,Wong2014,Ousterhout_presentation,Verissimo2001,correia2019byzantine,raft_paper}. We recommend students to conduct additional research on other sources \cite{mit,princeton}. %The reference section should be viewed as a ``additional readings reference'' - if you would like more information about the topic. @@ -100,219 +100,216 @@ \section*{Preliminary Notes} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT STARTS HERE -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Distributed Systems} \label{sec:ds} -A Distributed System is a system comprised of software and hardware components, that are connected on a network, coordinating their actions via message exchange. +A Distributed System is a system comprised of software and hardware components that are connected on a network, coordinating their actions via message exchange. \begin{figure}[h!] - \centering - \includegraphics[scale=0.25]{figures/itsystem.png} - \caption{Enterprise IT system \cite{Wong2014}} - \label{fig:internet} +\centering +\includegraphics[scale=0.25]{figures/itsystem.png} +\caption{Enterprise IT system \cite{Wong2014}} +\label{fig:internet} \end{figure} -Nowadays, most modern enterprise information systems are distributed systems, popularized due to the growth of the Internet. +Nowadays, most modern enterprise information systems are distributed systems, popularized due to the Internet's growth. Servers communicate with other servers using protocols, such as the HTTP protocol, based on the TCP/IP protocol. Communication can also happen locally (i.e., between computer processes, using interfaces such as sockets). Typically, such protocols are independent of the system architecture. In fact, there are different architectures for organizing distributed systems. One of the simplest is the client-server architecture, where servers can expose endpoints to clients. -In this architecture, servers keep resources and serve those resources according to requests made by clients. Client-server architectures have a popularized programming model: RPC, and the more recent gRPC. Such protocols allow to structure distributed system programming, based on client calls on server's code. While this is an efficient scheme, it constitutes a centralized paradigm, that suffers from issues such as lack of scalability, and constitutes a single point of failure. Other architectures offer different tradeoffs: for instance, the peer-to-peer architecture is decentralized, which allows for a higher degree of openness, and fault tolerance. This makes the whole system more robust to faults. +In this architecture, servers keep resources and serve those resources according to requests made by clients. Client-server architectures have a popularized programming model: RPC, and the more recent gRPC. Such protocols allow structuring distributed system programming based on client calls on the server's code. While this is an efficient scheme, it constitutes a centralized paradigm that suffers from a lack of scalability and constitutes a single point of failure. Other architectures offer different tradeoffs: for instance, the peer-to-peer architecture is decentralized, which allows for a higher degree of openness and fault tolerance. This makes the whole system more robust to faults. \subsection{Faults, Errors, and Failures} -A fault is an event that alters the normal behaviour of a system, which typically causes errors. Errors comprise the transition from a correct state of the system to an incorrect state. Errors can provoke failures, if the system deviates from its specification. The protection against faults allows participants to remain consistent: this way it is easier to maintain a correct state, that can be shared with other participants. Faults can be defined as crash faults or byzantine faults. Crash faults are faults where a node becomes unresponsive. This can be due to either physical problems, like a power outage or a faulty component, or due to networking problem, like network partitions which lead to the node being unreachable. Byzantine faults occur when a node acts arbitrarily and strays from the specification. This can include fake information, different responses to different nodes or even a completely random answer. +A fault is an event that alters the expected behavior of a system, which typically causes errors. Errors comprise the transition from a correct state of the system to an incorrect state. Errors can provoke failures if the system deviates from its specification. The protection against faults allows participants to remain consistent: this way, it is easier to maintain a correct state shared with other participants. Faults can be defined as crash faults or byzantine faults. Crash faults are faults where a node becomes unresponsive. This can be due to either physical problems, like a power outage or a faulty component, or networking problem, like network partitions, leading to the node being unreachable. Byzantine faults occur when a node acts arbitrarily and strays from the specification. This can include fake information, different responses to different nodes, or even a completely random answer. \subsection{Fault Tolerant Models} -There are different ways to deal with each type of faults. -Crash Fault Tolerant (CFT) models can usually withstand up to $\frac{N}{2}$ crashes, with N being the total number of nodes. This model assumes a quorum of $\frac{n}{2}+1$ nodes which has to agree on the global state. This means that as long as there is a \textbf{majority of nodes operational}, a consistent answer can be returned. +There are different ways to deal with each type of fault. +Crash Fault Tolerant (CFT) models can usually withstand up to $\frac{N}{2}$ crashes, with N being the total number of nodes. This model assumes a quorum of $\frac{n}{2}+1$ nodes, which has to agree on the global state. This means that as long as there is a \textbf{majority of nodes operational}, a consistent answer can be returned. -However, if one of the nodes is malicious, the system may enter a faulty (or inconsistent) state with the node giving fake information. As such, for Byzantine Fault Tolerant (BFT) models, we need at least $3f+1$ correct nodes to support $f$ malicious nodes. This way, we can guarantee that in any given quorum, we have at least a \textbf{majority of correct nodes} answers. +However, suppose one of the nodes is malicious. In that case, the system may enter a faulty (or inconsistent) state with the node giving fake information. As such, for Byzantine Fault Tolerant (BFT) models, we need at least $3f+1$ correct nodes to support $f$ malicious nodes. This way, we can guarantee that we have at least a \textbf{majority of correct nodes} answers in any given quorum. -%figure client server vs distributed - -But a question may arise: how to make sure that, in decentralized networks, the participants agree on a single state? How can we assure agreement under uncertain conditions, and assuming that participants can fail (e.g., connection problems)? What if there are malicious participants? This problem is also referred as the consensus problem, a well known problem studied the early stages of distributed systems. +But a question may arise: how to make sure that the participants agree on a single state? How can we assure agreement under uncertain conditions and assuming that participants can fail (e.g., connection problems)? What if there are malicious participants? This problem is also referred to as the consensus problem, a well-known problem studied in the early stages of distributed systems. \subsection{Consensus} -Consensus is a fundamental problem in distributed systems, in the area of fault tolerance. Consensus involves multiple servers (or nodes) agreeing on a global state (set of values). %The problem of consensus has been studied since the late 1970s, . -The problem of reaching consensus was popularized by Pease, Shostak, and Lamport's Byzantine generals story, where the generals had to agree on a common plan to attack a city, where the messengers that exchanged their messages where unreliable \cite{byzantine_generals}. +Consensus is a fundamental problem in distributed systems, in the area of fault tolerance. It involves multiple servers (or nodes) agreeing on a global state (set of values). + +Pease, Shostak, and Lamport's Byzantine generals story popularized the problem of reaching consensus, where the generals had to agree on a common plan to attack a city. In this scenario, messengers that exchanged their messages were unreliable \cite{byzantine_generals}, because they could be captured or change sides. Distributed systems ought to be \emph{Byzantine fault-tolerant}, as there may be malicious nodes on the network \cite{correia2019byzantine}. A consensus algorithm is used to create agreement on a global ledger state in the presence of crash faults or Byzantine faults. Consensus algorithms depend on the assumptions made about the environment and the system that the algorithm is running. For instance, what are the faults that can happen (crash vs byzantine), how are messages passed, synchronous (communication and processing delays are bounded) vs asynchronous (communication and processing delays are not bounded, i.e., may take an arbitrary amount of time to be completed) environment. \subsection{State Machine Replication} -A general approach to building fault-tolerant systems is state machine replication, that is studied in the context of consensus. A state machine can be described by a set of possible initial states and next relation. This relation determines the possible steps, given the current step. +A general approach to building fault-tolerant systems is state machine replication. A state machine can be described by a set of possible initial states, accepting states, and a transition function. This function determines the possible steps, given the current state. -In a distributed system, each participating node has a state machine and a log. A log is a collection of log entries, which typically contain timestamped information. The state machine is a device (computer) that stores and manipulates state (e.g., values written in an hash table, the value of a variable, an array with commands to be executed). +In a distributed system, each participating node has a state machine and a log. A log is a collection of log entries, which typically contain timestamped information. The state machine (for example, a computer program) stores and manipulates state (for example, the state can be values are written in a hash table or an array with commands to be executed). -The idea of state machine replication is that clients can interact with a distributed system that agrees upon a global state, while it appears that it is interacting with a single state machine. Even if one of the server fails, the system still responds to the requests. Each node takes commands from the logs, triggering the execution of a function. The consensus ensures that the commands received are the same while ensuring a common ordering for the commands received. This ensures that each state machine yields the same series of results, achieving the same series of states, at a common order. Each state machine executes the same function calls, in the same order, yielding the same output. For this reason, state machines need to be deterministic (otherwise, the same input could yield different outputs, causing inconsistencies in the system). - - \begin{figure}[h!] - \centering - \includegraphics[scale=0.25]{figures/rsm.png} - \caption{Replicated state machines \cite{replicated}} - \label{fig:rms} +The idea of state machine replication is that clients can interact with a distributed system that agrees upon a global state while it appears that it is interacting with a single state machine. Even if one of the servers fails, the system still responds to the requests. Each node takes commands from the logs, triggering the execution of a function. The consensus ensures that the commands received are the same while ensuring a common ordering for the commands. This ensures that each state machine yields the same series of results, achieving the same series of states, in a common order. Each state machine executes the same function calls, in the same order, yielding the same output. For this reason, state machines need to be deterministic (otherwise, the same input could produce different results, causing inconsistencies in the system). + +\begin{figure}[h!] +\centering +\includegraphics[scale=0.25]{figures/rsm.png} +\caption{Replicated state machines \cite{replicated}} +\label{fig:rms} \end{figure} -Figure \ref{fig:rms} shows an example of a replicated state machine. A group of machines executes a deterministic function based on a replicated log they receive (input). The idea is that logs are kept identical so the execution on different machines is performed using the same commands, and on the same order. When the first machine receives an update request (set variable Z to 6), it first replicates that command into all the logs, passing that command to the other machines on the cluster. All machines add the newest command to the end of their logs. After that, the command can be executed by each state machine. This triggers the original state machine to return the answer of the execution to the client. If one of the servers (that comprises a state machine) fails, the other servers can still process the request, as far as a majority of servers are operational. There are consensus algorithms that apply either Crash Fault Tolerant or Byzantine Fault Tolerant models. These systems allow to develop and deliver resilient, dependable systems that can satisfy today's needs in terms of availability and scalability. +Figure \ref{fig:rms} shows an example of a replicated state machine. A group of machines executes a deterministic function based on a replicated log they receive (input). The idea is that logs are kept identical, so the execution on different machines is performed using the same commands and in the same order. When the first machine receives an update request (set variable Z to 6), it first replicates that command into all the logs, passing that command to the other machines on the cluster. All machines add the newest command to the end of their logs. After that, the command can be executed by each state machine. This triggers the original state machine to return the answer to the execution to the client. If one of the servers (that comprises a state machine) fails, the other servers can still process the request, as far as a majority of servers are operational. There are consensus algorithms that apply either Crash Fault Tolerant or Byzantine Fault Tolerant models. These systems allow us to develop and deliver resilient, dependable systems that can satisfy today's needs in terms of availability and scalability. + +With this in mind, consensus algorithms aim to replicate commands into the replicated logs, allowing us to assure desirable properties about a system, like validity, agreement, and termination. -With this in mind, consensus algorithms then aim to replicate commands into the replicated logs, allowing us to assure desirable properties about a system, like validity, agreement and termination. +A system is valid if the value decided by the system was a value proposed by a node. This way, we avoid cases where a node can inject a decision that no one has seen. Moreover, a system agrees if no two correct processes decide differently (e.g., output different values for the same input). Termination is a liveness property that states ``every correct process eventually decide``. Those are all desirable properties for fault-tolerant systems. -A system is valid if the value decided by the system was a value proposed by a node. This way, we avoid cases where a node can inject a decision that no one has seen. Moreover, a system is in agreement if no two correct processes decide differently (e.g., output different values for the same input). Termination is a liveness property, that states ``every correct process eventually decide``. Those are all desirable properties for fault-tolerant systems. +In this class, you will work in greater detail the RAFT algorithm: a well-adopted consensus algorithm, which focuses on understandability and has many practical applications, inclusively on the blockchain area. -In this class, you will work in greater detail the RAFT algorithm: a well-adopted consensus algorithm, which focuses on understandability, and has many practical applications, inclusively on the blockchain area. - \subsection{The RAFT consensus algorithm} -RAFT is a crash fault tolerant consensus algorithm that is designed to be easy to understand. %Typically, crash fault tolerant leader-based algorithms have a linear message complexity, O(n), as opposed to Byzantine leader-based consensus, which incurrs in a quadratic message complexity (On2). +RAFT is a crash fault-tolerant consensus algorithm that is designed to be easy to understand. It belongs to a class of consensus algorithms called leader-based (or coordinator-based or primary-backup). In leader-based consensus, a leader calculates a decision on the state and communicates it to the followers; if the leader is faulty, a new leader is elected. -It belongs to a class of consensus algorithms called leader-based (or coordinator-based or primary-backup). In leader-based consensus, there is a leader that calculates a decision on the state and communicates it to the followers; if the leader is faulty, a new leader is elected. -RAFT considers several participants: +RAFT considers several participants: \begin{itemize} - \item Leader: the node that receives client requests, manages the log replication, and sends log update requests to other nodes. While having a leader that commands the global state is sufficient fine in some scenarios, a malicious leader can constitute a problem, as eventual inconsistencies are corrected (overwritten) by the leader. - \item Followers: constituting all nodes except the leader; - \item Candidates: followers requesting to be the leader. +\item leader: the node that receives client requests, manages the log replication and sends log update requests to other nodes. While having a leader that commands the global state is sufficient fine in some scenarios, a malicious leader can constitute a problem; +\item Followers: constituting all nodes except the leader; +\item Candidates: followers requesting to be the leader. \end{itemize} -As explained before, a distributed system built with RAFT has several servers running an instance of RAFT and a persistent log to store the commands received from clients. The replication of logs and consequent command execution is given by three phases: +As explained before, a distributed system built with RAFT has several servers running a RAFT instance and a persistent log to store the commands received from clients. The replication of logs and consequent command execution is given by three phases: \begin{enumerate} - \item Leader Election - \item Log Replication - \item Safety +\item Leader Election +\item Log Replication +\item Safety \end{enumerate} -Figure \ref{fig:state_empty} shows the global state, which corresponds to the state of each node (server), across the different terms (time periods in which the algorithm executes). Terms are used to detect obsolete information: there is at most one leader per term, and followers do not accept information coming from a previous term than the one they record. This way, the algorithm guarantees that each node has the latest information. +Figure \ref{fig:state_empty} shows the global state, which corresponds to the state of each node (server) across the different terms (time periods in which the algorithm executes). Terms are used to detect obsolete information: there is at most one leader per term, and followers do not accept information coming from a previous term than the one they record. This way, the algorithm guarantees that each node has the latest information. \begin{figure} - \centering - \subfloat[Initial state of the shared log of operations]{ \label{fig:state_empty}{\includegraphics[width=7cm]{figures/RAFT_state_empty.png} }}% - \qquad - \subfloat[Leader S2 received a command from a client. It wishes to updated state]{ \label{fig:RAFT5}{\includegraphics[width=7cm]{figures/RAFT_state.png} }}% - \caption{States at different terms for each server}% - \label{fig:state_empty}% +\centering +\subfloat[Initial state of the shared log of operations]{ \label{fig:state_empty}{\includegraphics[width=7cm]{figures/RAFT_state_empty.png} }}% +\qquad +\subfloat[Leader S2 received a command from a client. It wishes to updated state]{ \label{fig:RAFT5}{\includegraphics[width=7cm]{figures/RAFT_state.png} }}% +\caption{States at different terms for each server}% +\label{fig:state_empty}% \end{figure} - \subsubsection{Leader Election} -Time in RAFT is divided into terms. Each terms has a unique identifier, a monotonically increasing number, which is set to 0 at server startup time. RAFT uses randomized timers to elect the leader, illustrated by Figure \ref{fig:le_1}. In order to know if the leader is alive, RAFT uses an heartbeat system. At the beginning of the consensus process, nodes are awaiting for heartbeats from the leader (Figure \ref{fig:RAFT1}). When a follower receives a heartbeat, their timeouts are reset. As there is no leader, no heartbeats are received within the election timeout, and eventually a timeout is triggered. The node that triggered the timeout sends a proposal to become the new leader to every other node (Figure \ref{fig:RAFT2}), and eventually becomes the new leader. +Time in RAFT is divided into terms. Each term has a unique identifier, a monotonically increasing number set to 0 at server startup time. RAFT uses randomized timers to elect the leader, illustrated in Figure \ref{fig:le_1}. To know if the leader is alive, RAFT uses a heartbeat system. At the beginning of the consensus process, nodes are awaiting heartbeats from the leader (Figure \ref{fig:RAFT1}). When a follower receives a heartbeat, their timeouts are reset. As there is no leader, no heartbeats are received within the election timeout, and eventually, a timeout is triggered. The node that initiated the timeout sends a proposal to become the new leader to every other node (Figure \ref{fig:RAFT2}), and eventually becomes the new leader. -If a node does not receive a heartbeat from a leader within a certain time frame (for example, in case the leader crashes), there is another timeout that initiates the beginning of a new term and a new leader election process starts. +Suppose a node does not receive a heartbeat from a leader within a certain time frame (for example, in case the leader crashes). In that case, there is another timeout that initiates the beginning of a new term, and a new leader election process starts. \begin{figure} - \centering - \subfloat[There is no leader, each node awaits instructions]{ \label{fig:RAFT1}{\includegraphics[width=7cm]{figures/RAFT1.png} }}% - \qquad - \subfloat[After the timeout, the node sends a proposal to become the new leader]{ \label{fig:RAFT2}{\includegraphics[width=7cm]{figures/RAFT2.png} }}% - \caption{Leader election process}% - \label{fig:le_1}% +\centering +\subfloat[There is no leader, each node awaits instructions]{ \label{fig:RAFT1}{\includegraphics[width=7cm]{figures/RAFT1.png} }}% +\qquad +\subfloat[After the timeout, the node sends a proposal to become the new leader]{ \label{fig:RAFT2}{\includegraphics[width=7cm]{figures/RAFT2.png} }}% +\caption{Leader election process}% +\label{fig:le_1}% \end{figure} -The terms also allows for liveness: nodes reject requests coming with an invalid term number (older than the current term). Nodes ask to be a leader in a first-come-first-served basis. As the majority rule applies, only one node can win an election and become the leader. In case of a draw, another round of election proceeds. Note that at the beginning of the execution, the log is empty (Figure \ref{fig:state_empty}). +The terms also allow for liveness: nodes reject requests coming with an invalid term number (older than the current term). Nodes ask to be a leader on a first-come-first-served basis. As the majority rule applies, only one node can win an election and become the leader. In case of a draw, another round of election proceeds. Note that the log is empty at the beginning of the execution (Figure \ref{fig:state_empty}). \subsubsection{Log Replication} -The idea of log replication is the leader to accept requests from clients, which translate to commands that the server has to execute. Those commands are appended to a log. After that, the leader replicates its logs to other nodes on the network, so they can execute the received commands as well, in order to keep the states of all nodes consistent. Part of this procedure is illustrated by Figure \ref{fig:su_2}. +The idea of log replication is the leader to accept requests from clients, which translates to commands that the server has to execute. Those commands are appended to a log. After that, the leader replicates its logs to other nodes on the network to execute the received commands and keep the states of all nodes consistent. Part of this procedure is illustrated in Figure \ref{fig:su_2}. -Let us now suppose that a leader receives a request from a client (Figure \ref{fig:RAFT_state}). The log is updated, but it is still not safe to execute the received commands (as the log entry is considered uncommitted). This happens because followers are not aware of the request (illustrated by the dashed lines around an update). The leader broadcasts the most recent log entry to their followers, and receives a message from each one acknowledging that they received the updated log entry (Figure \ref{fig:RAFT6}). After the leader confirms the state, after receiving the acknowledgments from the followers, the followers can append the command to their logs, for later executing the command. +Let us now suppose that a leader receives a request from a client (Figure \ref{fig:RAFT_state}). The log is updated, but it is still not safe to execute the received commands (as the log entry is considered uncommitted). This happens because followers are not aware of the request (illustrated by the dashed lines around an update). The leader broadcasts the most recent log entry to their followers and receives a message from each one acknowledging that they received the updated log entry (Figure \ref{fig:RAFT6}). After the leader confirms the state, after receiving the followers' acknowledgments, the followers can append the command to their logs, for later executing the command. \begin{figure}[h!] - \centering - \subfloat[Followers acknowledge the latest log entry]{ \label{fig:RAFT_state}{\includegraphics[width=7cm]{figures/RAFT5.png} }}% - \qquad - \subfloat[Global status is committed, that is, the received command from the client is present in the logs of the majority of nodes, so each node can now execute it]{ \label{fig:RAFT6}{\includegraphics[width=7cm]{figures/RAFT6.png} }}% - \caption{State updates}% - \label{fig:su_2}% +\centering +\subfloat[Followers acknowledge the latest log entry]{ \label{fig:RAFT_state}{\includegraphics[width=7cm]{figures/RAFT5.png} }}% +\qquad +\subfloat[Global status is committed, that is, the received command from the client is present in the logs of the majority of nodes, so each node can now execute it]{ \label{fig:RAFT6}{\includegraphics[width=7cm]{figures/RAFT6.png} }}% +\caption{State updates}% +\label{fig:su_2}% \end{figure} -Imagine that now, node S2 crashes due to a hardware fail. This causes that S2 cannot send heartbeats back to each follower. We now assume that S1 timeouts - it becomes a candidate and tries to become a leader (Figure \ref{fig:crash}). When a follower becomes a candidate, it increments the current term, votes for itself, and send requests to other followers, to become the new leader. Eventually, S1 wins the election and becomes the new leader (Figure \ref{fig:new_leader}). +Imagine that now, node S2 crashes due to a hardware failure. This causes that S2 cannot send heartbeats back to each follower. We now assume that S1 timeouts - it becomes a candidate and tries to become a leader (Figure \ref{fig:crash}). When a follower becomes a candidate, it increments the current term, votes for itself, and send requests to other followers to become the new leader. Eventually, S1 wins the election and becomes the new leader (Figure \ref{fig:new_leader}). \begin{figure}[h!] - \centering - \subfloat[Server S2 crashes. A follower triggers a timeout, becoming a candidate. The candidate S1 tries to become the leader]{ \label{fig:crash}{\includegraphics[width=7cm]{figures/s2_crash.png} }}% - \qquad - \subfloat[Candidate S1 becomes the leader]{ \label{fig:new_leader}{\includegraphics[width=7cm]{figures/new_leader.PNG} }}% - \caption{New leader election process}% - \label{fig:leader_election}% +\centering +\subfloat[Server S2 crashes. A follower triggers a timeout, becoming a candidate. The candidate S1 tries to become the leader]{ \label{fig:crash}{\includegraphics[width=7cm]{figures/s2_crash.png} }}% +\qquad +\subfloat[Candidate S1 becomes the leader]{ \label{fig:new_leader}{\includegraphics[width=7cm]{figures/new_leader.PNG} }}% +\caption{New leader election process}% +\label{fig:leader_election}% \end{figure} \subsubsection{Safety} -Logs need to be kept consistent, and, furthermore, that only nodes with the latest log version can become leaders. +Logs need to be kept consistent, and, furthermore, only nodes with the latest log version can become leaders. If a crash occurs, log inconsistencies may happen. In other words, the information stored by each node may not be the same at the same point. To solve this problem, the log of the current leader is always considered the correct version. The leader repairs inconsistencies, if applicable. For that, there cannot be missing log entries, and log entries need to be ordered. -RAFT's approach to maintain consistency is by matching logs and solving conflicts: each time a leader sends a log entry to a follower, it includes the term number and log index of the preceding entry. If there is a match, the follower accepts the new log entry; otherwise, the follower rejects the log entry. In case a log entry is rejected, the leader tries again, sending the current log entry, the preceding one, and the one further before., until a match happens. When a match happens, the follower's log is updated, accordingly to the leader. +RAFT's approach to maintaining consistency is by matching logs and solving conflicts: each time a leader sends a log entry to a follower, it includes the term number and log index of the preceding entry. If there is a match, the follower accepts the new log entry; otherwise, the follower rejects the log entry. In case a log entry is rejected, the leader tries again, sending the current log entry, the preceding one, and the one further before., until a match happens. When a match happens, the follower's log is updated, according to the leader. -The explained mechanisms guarantee the log matching property: if a given entry is committed, all previous entries are committed and, if two logs share the same log index and term, then those logs are identical (represent the same log entry). Lastly, the desirable safety tackles leader completeness: once a log entry is committed, the leader will have that log entry stored. This assures a committed entry is never lost. Nodes with incomplete logs are not chosen in the election process. Essentially, a node denies the vote if their log is more complete than the candidate's log. +The explained mechanisms guarantee the log matching property: all previous entries are committed if a given entry is committed. If two logs share the same log index and term, then those logs are identical (represent the same log entry). Lastly, the desirable safety tackles leader completeness: once a log entry is committed, the leader will have that log entry stored. This assures a committed entry is never lost. Nodes with incomplete logs are not chosen in the election process. Essentially, a node denies the vote if their log is more complete than the candidate's log. \section{Hands on RAFT} -Now, that we illustrated the RAFT algorithm, let's consolidate that knowledge. It is recommended for students to read the additional support materials available: \cite{raft_paper,raft_homepage}. You may use the simulator provided at RAFT's homepage to aid you in your responses\footnote{https://RAFT.github.io/}. +Now that we illustrated the RAFT algorithm, let's consolidate that knowledge. It is recommended for students to read the additional support materials available: \cite{raft_paper,raft_homepage}. You may use the simulator provided at RAFT's homepage to aid you in your responses\footnote{https://RAFT.github.io/}. -\subsubsection*{Exercise 1:} To consolidate your knowledge, watch the guided visualization of the RAFT algorithm \cite{raft_viz}. -Elaborate on how the RAFT algorithm can contribute to increase the availability of the system, from a client perspective. +\subsubsection*{Exercise 1:} To consolidate your knowledge, watch the guided visualization of the RAFT algorithm \cite{raft_viz}. +Elaborate on how the RAFT algorithm can increase the system's availability from a client perspective. -\subsubsection*{Exercise 2:} When does a new term start? +\subsubsection*{Exercise 2:} When does a new term start? -\subsubsection*{Exercise 3:} How does candidates act if two of them attempt to become leaders at exactly the same time? +\subsubsection*{Exercise 3:} How do candidates act if two of them attempt to become leaders at precisely the same time? -\subsubsection*{Exercise 4:} If a follower receives a message has a different term than the one recorded, does it accept the message? +\subsubsection*{Exercise 4:} If a follower receives a message that has a different term than the one recorded, does it accept the message? -\subsubsection*{Exercise 5:} Suppose that a leader receives a command from a client, and notifies all followers. Upon sending the second transaction, one of the followers crashes. How does RAFT guarantee that the follower is updated? +\subsubsection*{Exercise 5:} Suppose that a leader receives a command from a client and notifies all followers. Upon sending the second transaction, one of the followers crashes. How does RAFT guarantee that the follower is updated? -\subsubsection*{Exercise 6:} Consider the following RAFT cluster, composed by a leader and four followers. The leader has sent eight messages, corresponding to eight commands. Those eight commands translate into eight log entries. The current term is term three. +\subsubsection*{Exercise 6:} Consider the following RAFT cluster, composed by a leader and four followers. The leader has sent eight messages, corresponding to eight commands. Those eight commands translate into eight log entries. The current term is term three. \begin{figure}[H] - \centering - \includegraphics[scale=0.6]{figures/exercise.png} - \caption{RAFT cluster composed of 5 machines} - \label{fig:RAFT_cluster} +\centering +\includegraphics[scale=0.6]{figures/exercise.png} +\caption{RAFT cluster composed of 5 machines} +\label{fig:RAFT_cluster} \end{figure} What are the commands, represented by each log index, that are safe to be executed by each node? -\subsubsection*{Exercise 7:} Consider the following RAFT cluster, composed by a leader and four followers, in which there are inconsistencies on the logs across nodes. Assume that node S4 crashed on term 1. Node S5 became the leader on term 2, but was only successful to replicate logs 4 and 5, before crashing. Node S3 is the leader on term 3. +\subsubsection*{Exercise 7:} Consider the following RAFT cluster, composed of a leader and four followers. There are inconsistencies in the logs across nodes. Assume that node S4 crashed on term 1. Node S5 became the leader on term 2, but was only successful in replicating logs 4 and 5, before crashing. Node S3 is the leader in term 3. \begin{figure}[H] - \centering - \includegraphics[scale=0.6]{figures/inconsistent_logs.png} - \caption{RAFT cluster composed of 5 servers, showing inconsistencies regarding the log} - \label{fig:inconsistent_logs} +\centering +\includegraphics[scale=0.6]{figures/inconsistent_logs.png} +\caption{RAFT cluster composed of 5 servers, showing inconsistencies regarding the log} +\label{fig:inconsistent_logs} \end{figure} -What will be the final state of the system, assuming no new commands are received, and that S5 recovers from crash? +What will be the system's final state, assuming no new commands are received, and that S5 recovers from a crash? -\subsubsection*{Exercise 8:} During the leader election process, a mechanism that ensure consistency across logs is the rejection of candidates that have outdated logs. What are the checks the follower does against the candidate's proposal? In which occasion does it reject voting on the candidate? +\subsubsection*{Exercise 8:} During the leader election process, a mechanism that ensures consistency across logs is the rejection of candidates with outdated logs. What are the checks the follower does against the candidate's proposal? On which occasion does it reject voting on the candidate? -\subsubsection*{Exercise 9:} Consider the following RAFT cluster, composed by seven nodes. +\subsubsection*{Exercise 9:} Consider the following RAFT cluster, composed of seven nodes. \begin{figure}[H] - \centering - \includegraphics[scale=0.6]{figures/discussion.png} - \caption{RAFT cluster composed of 7 nodes} - \label{fig:inconsistent_logs} +\centering +\includegraphics[scale=0.6]{figures/discussion.png} +\caption{RAFT cluster composed of 7 nodes} +\label{fig:inconsistent_logs} \end{figure} -Given that the leader of term eight has ten log entries, and interacts with six followers, how did the system get into this state? +Given that the leader of term eight has ten log entries and interacts with six followers, how did the system get into this state? \subsubsection*{Exercise 10:} How does the system re-organize itself if a node leaves the system? diff --git a/source/Lab01/Instructors Guide/assignment.tex b/source/Lab01/Instructors Guide/assignment.tex index d99c07b..6feb84c 100644 --- a/source/Lab01/Instructors Guide/assignment.tex +++ b/source/Lab01/Instructors Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -17,8 +17,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -68,16 +68,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 1 \\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & 14 Sept 2020 \\ +\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Background: Distributed Systems} & \textbf{Due Date:} & 21 Sept 2020\\ +\multicolumn{2}{|l|}{Background: Distributed Systems} & \textbf{Due Date:} & \\ \hline \end{tabularx}} \rhead{} @@ -92,63 +92,61 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 \section*{Instructors Guide} -This document provides a proposal of solution for LAB\#01, which concerns the an introduction to distributed systems and the RAFT consensus algorithm. Most of the solutions can be found at \cite{raft_paper, Ousterhout_presentation, raft_viz,raft_homepage}. +This document provides a proposal for a solution for LAB\#01, which concerns an introduction to distributed systems and the RAFT consensus algorithm. Most of the solutions can be found at \cite{raft_paper, Ousterhout_presentation, raft_viz,raft_homepage}. -This class aims to illustrate what is consensus, and provide an intuition on how does it work. For that, we present the RAFT algorithm. Hoewver, this class does not present RAFT in an exhaustive manner. For details, check the paper \cite{raft_paper}. +This class aims to illustrate what consensus is and provide intuition on how does it work. For that, we present the RAFT algorithm. However, this class does not present RAFT in a thorough manner. For details, check the paper \cite{raft_paper}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT STARTS HERE -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{RAFT} \label{sec:RAFT} -\subsubsection*{Exercise 1: To consolidate your knowledge, watch the a guided visualization of the RAFT algorithm \cite{raft_viz}. -Elaborate on how the RAFT algorithm can contribute to eliminate failures from a client perspective.} +\subsubsection*{Exercise 1:} To consolidate your knowledge, watch the guided visualization of the RAFT algorithm \cite{raft_viz}. +Elaborate on how the RAFT algorithm can increase the system's availability from a client perspective. -RAFT is a crash fault tolerant consensus algorithm - meaning that it tolerates when some servers in a cluster crash. When a client makes a request to the system, the leader disseminates the request, making every follower execute it. If a fault occurs in one of the followers, leading to an inconsistent state (error). Assuming that such error leads to a failure, and the system crashes, the resiliency of the whole the system as a whole will not probably fail, as that fault is tolerated by other nodes. This way, the system is more likely to be available. +RAFT is a crash fault-tolerant consensus algorithm that it tolerates when some servers are in a cluster crash. When a client requests the system, the leader disseminates the request, making every follower execute it. If a fault occurs in one of the followers, leading to an inconsistent state (error). Assuming that such error leads to a failure and the system crashes, the whole system's resiliency will not probably fail, as that fault is tolerated by other nodes. This way, the system is more likely to be available. -\subsubsection*{Exercise 2: When does a term start? } -A new term starts whenever a new leader is elected, or if there is a tie at the leader election process. +\subsubsection*{Exercise 2:} When does a new term start? +A new term starts whenever a new leader is elected or a tie at the leader election process. +\subsubsection*{Exercise 3:} How do candidates act if two of them attempt to become leaders at precisely the same time? -\subsubsection*{Exercise 3: How does candidates act if two of them attempt to become leaders at exactly the same time?} -When votes are split in a way there is no majority of any candidate, the outcome of a leader election is ``(c) a period of time goes by with no winner'' \cite{raft_paper}. In this case, each candidate will time out and start a new election. A new election includes incrementing the term number and to initiate a new round of voting. There is at most one winner per term; and candidates eventually win, as the new timeouts are chosen randomly. +When votes are split in a way, there is no majority of any candidate, the outcome of a leader election is ``(c) a period of time goes by with no winner'' \cite{raft_paper}. In this case, each candidate will time out and start a new election. A new election includes incrementing the term number and to initiate a new round of voting. There is at most one winner per term, and candidates eventually win, as the new timeouts are chosen randomly. -\subsubsection*{Exercise 4: If a follower receives a message has a different term than the one recorded, does it accept the message? } +\subsubsection*{Exercise 4:} If a follower receives a message that has a different term than the one recorded, does it accept the message? -If a message received has a greater term than the term recorded by a node, that node updates its term and becomes a follower. If a message received has a lower term than the recorded . +If a message received has a greater term than the term recorded by a node, that node updates its term and becomes a follower. If a message received has a lower term than the recorded. %\subsubsection*{Exercise 5: Explain why a client cannot obtain inconsistent messages, depending on the machine that is queried.} -\subsubsection*{Exercise 5: Suppose that a leader receives a command from a client, and notifies all followers. Upon sending the second transaction, one of the followers crashes. How does RAFT guarantee that the follower is updated?} +\subsubsection*{Exercise 5:} Suppose that a leader receives a command from a client and notifies all followers. Upon sending the second transaction, one of the followers crashes. How does RAFT guarantee that the follower is updated? + +After the leader receives a command, the leader propagates that new command information to the followers (uncommitted). Only then he executes the function and returns the result to the client. The followers are then notified, and they execute the command in their state machine. The transaction is considered committed at this point. The follower is updated by following the Log matching property. +\subsubsection*{Exercise 6: Consider the following RAFT cluster, composed of a leader and four followers. The leader has sent eight messages, corresponding to eight commands. Those eight commands translate into eight log entries. The current term is term three. What are the commands, represented by each log index, that are safe to be executed by each node?} -After the leader receives a command, the leader propagates that information of a new command to the followers (uncommitted). Only then he executes the function and returns the result to the client. The followers are then notified, and they execute the command in their state machine. The transaction is considered committed at this point. The follower is updated through following the the Log matching property. +The log entries that are considered to be committed are log entries up to log index 7, because they are replicated across most nodes (3 out of 5). Two servers are out of sync. -\subsubsection*{Exercise 6: Consider the following RAFT cluster, composed by a leader and four followers. The leader has sent eight messages, corresponding to eight commands. Those eight commands translate into eight log entries. The current term is term three. What are the commands, represented by each log index, that are safe to be executed by each node?} +\subsubsection*{Exercise 7:} Consider the following RAFT cluster, composed of a leader and four followers. There are inconsistencies in the logs across nodes. Assume that node S4 crashed on term 1. Node S5 became the leader on term 2, but was only successful in replicating logs 4 and 5, before crashing. Node S3 is the leader in term 3. -The log entries that are consider to be committed are log entries up to log index 7, because they are replicated across the majority of nodes (3 out of 5). Two servers are out of sync. +RAFT assumes that the current leader is correct (holds the most up-to-date log of commands). S5 will discard its version of logs 6-9, and receive the updated log from S3. S1 and S2 eventually receive logs 8 and 9. As most nodes have the log complete up to log entry 9, the commands present in the log entries can be executed. -\subsubsection*{Exercise 7: Consider the following RAFT cluster, composed by a leader and four followers, in which there are inconsistencies on the logs across nodes. Assume that node S4 crashed on term 1. Node S5 became the leader on term 2, but was only successful to replicate logs 4 and 5, before crashing. Node S3 is the leader on term 3. What will be the final state of the system, assuming no new commands are received, and that S5 recovers from crash?} +\subsubsection*{Exercise 8: During the leader election process, a mechanism that ensures consistency across logs is the rejection of candidates that have outdated logs. What are the checks the follower does against the candidate's proposal? On which occasion does it reject voting on the candidate?} -In RAFT, it is assumed that the current leader is correct (holds the most up-to-date log of commands). S5 will discard its version of logs 6-9, and receive the updated log from S3. S1 and S2 eventually receive logs 8 and 9. As the majority of nodes have the log complete up to log entry 9, the commands present in the log entries can be executed. +The node checks if the candidate's last log entry is in the same term as the candidate's last log entry. If the node's last log entry is in a superior term, it does not vote on the candidate. If the terms are the same, and the node's log index is higher (more complete log), it does not vote on the candidate. This process certifies the elected leader has all the committed entries. -\subsubsection*{Exercise 8: During the leader election process, a mechanism that ensure consistency across logs is the rejection of candidates that have outdated logs. What are the checks the follower does against the candidate's proposal? In which occasion does it reject voting on the candidate?} +\subsubsection*{Exercise 9: Consider the following RAFT cluster, composed of seven nodes. Given that the leader of term eight has ten log entries and interacts with six followers, how did the system get into this state?} - The node checks if the last log entry from the candidate is in the same term as the candidate's last log entry. If the node's last log entry is in a superior term, it does not vote on the candidate. If terms are the same, and the node's log index is higher (more complete log), it does not vote on the candidate. This process certifies the elected leader has all the committed entries. - - \subsubsection*{Exercise 9: Consider the following RAFT cluster, composed by seven nodes. Given that the leader of term eight has ten log entries, and interacts with six followers, how did the system get into this state?} - - From RAFT's paper: When the leader at the top comes to power, it is -possible that any of scenarios (a–f) could occur in follower -logs. Each box represents one log entry; the number in the -box is its term. A follower may be missing entries (a–b), may +From RAFT's paper: When the leader at the top comes to power, it is +possible that any of the scenarios (a–f) could occur in follower +logs. Each box represents one log entry; the number in the box is its term. A follower may be missing entries (a–b), may have extra uncommitted entries (c–d), or both (e–f). For example, scenario (f) could occur if that server was the leader for term 2, added several entries to its log, then crashed before committing any of them; it restarted quickly, became leader @@ -159,8 +157,8 @@ \subsubsection*{Exercise 8: During the leader election process, a mechanism that \subsubsection*{Exercise 10: How does the system re-organize itself if a node leaves the system?} configuration changes must -use a two-phase approach: the first phase disables the old consortium configuration, disabling the processing of client requests; the second phase enables the new configuration. ``In Raft the cluster first switches to a transitional configuration we call joint -consensus; once the joint consensus has been committed, the system then transitions to the new configuration''. +use a two-phase approach: the first phase disables the old consortium configuration, disabling client requests' processing; the second phase enables the new configuration. ``In RAFT, the cluster first switches to a transitional configuration we call joint +consensus; once the joint consensus has been committed, the system then transitions to the new configuration''. For more details, see Section 6 of \cite{raft_paper}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT ENDS HERE diff --git a/source/Lab02/Guide/assignment.tex b/source/Lab02/Guide/assignment.tex index a94d1ef..cc36f22 100644 --- a/source/Lab02/Guide/assignment.tex +++ b/source/Lab02/Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -22,8 +22,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -52,7 +52,7 @@ commentstyle=\color{green}\itshape, moredelim=[s][\color{blue}\bfseries]{C:}{\>} } -\newcommand{\comment}[1]{} + \lstdefinestyle{Bash} {% language=bash, breaklines=true, @@ -73,16 +73,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 2 \\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & 21 Sept 2020 \\ +\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Background: Cryptography \& Security} & \textbf{Due Date:} & 28 Sept 2020\\ +\multicolumn{2}{|l|}{Background: Cryptography \& Security} & \textbf{Due Date:} & \\ \hline \end{tabularx}} \rhead{} @@ -97,7 +97,7 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 \section*{Preliminary Notes} -This class provides background about crpytography and security. This laboratory is based on several sources \cite{sdist2017,rogaway2004,md2020}. +This class provides background about cryptography and security. This laboratory is based on several sources \cite{sdist2017,rogaway2004,md2020}. %The reference section should be viewed as a ``additional readings reference'' - if you would like more information about the topic. @@ -105,165 +105,68 @@ \section*{Preliminary Notes} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT STARTS HERE -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Introduction} - -A big part of distributed systems is the communication between entities, which mostly happens over the internet. -This means that our communication is running trough channels open to the public that we as developers do not own nor have control over. -As such, we must build upon these insecure channels in order to obtain some useful properties, such as: - -\begin{itemize} - \item Confidentiality - \item Authentication - \item Non-Repudiation - \item Integrity - \item Freshness -\end{itemize} - -And many others. To that end, we use \textbf{Cryptographic primitives} in order to ensure this properties. We will see how we can implement each of these properties over the course of this lab. +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Cryptography and Security} -The goal of Cryptography is to allow secure communication between people. Whereas having a single dialect could serve that purpose, it is neither scalable or safe. - -Cryptography is much older than computers, dated back as early as 400 BC, where Spartans used a cipher device called \emph{scytale} for military communication \cite{simmons}. A \emph{cipher} is an algorithm that is used to obfuscate information (or to encrypt information). Information is \emph{encrypted} using a key, as the parameter of the cipher. This allows the algorithm to be public. Historical ciphers are, for example, the Caesar cipher\footnote{https://cryptii.com/pipes/caesar-cipher}, the substitution cipher\footnote{https://planetcalc.com/7984/}, the Vigener cipher\footnote{https://www.dcode.fr/vigenere-cipher}. +The goal of Cryptography is to allow communication between people or computers in a way that others cannot understand that communication. Whereas having a single dialect could serve that purpose, it is not scalable nor safe. +Cryptography is much older than computers, dated from as early as 400 BC, where Spartans used a cipher device called \emph{scytale} for military communication \cite{simmons}. A \emph{cipher} is an algorithm that is used to obfuscate information (or to encrypt information). Information is \emph{encrypted} using a key as the parameter of the cipher. This allows the algorithm to be public. Historical ciphers are, for example, the Caesar cipher\footnote{https://cryptii.com/pipes/caesar-cipher}, the substitution cipher\footnote{https://planetcalc.com/7984/}, the Vigener cipher\footnote{https://www.dcode.fr/vigenere-cipher}. - -\subsection{Confidentiality} - -The first main goal of cryptography was to hide information in order that only a selected few that knew a \textbf{secret} could read its contents. We call this \textbf{Confidentiality}. %Add the typical reference to WW2 and Enigma machine - -In this case, the secret used to hide information is called a key. Using the key to apply a certain operation to a message, we can alter the contents of the message in order to be incomprehensible to the common eye. To this incomprehensible message we call \textbf{Cipher}. - -We can then use the key again in order to obtain the original message. This way, \textbf{only the ones with access to the key} can understand the content of the message. - -Let us define notation to refer to ciphers, messages and their keys. +Let us define notation to refer to ciphers and messages and their keys. \theoremstyle{definition} \begin{definition}{Cryptogram C} -$C = E_k(M)$, where C is the cryptogram (or encrypted message), $E_k$ is the key used for the encryption, and M is the original message. +$C = E_k(M)$, where C is the cryptogram (or encrypted message), $E_k$ is the key used for the encryption, and M is the original message. \end{definition} \begin{definition}{Decrypted Message M} -$M = E_k^{-1}(C)$, where M is the decrypted message, $K^{-1}$ is the inverse key, and C is the cryptogram. +$M = E_k^{-1}(C)$, where M is the decrypted message, $K^{-1}$ is the inverse key, and C is the cryptogram. \end{definition} -In the context of cryptography, often techniques are separated into symmetric and asymmetric cryptography. Each type can give us different guarantees depending on our goal. We will start by looking at Symmetric Cryptography. +In the context of cryptography, often, techniques are separated into symmetric and asymmetric cryptography. -\subsubsection{Symmetric Cryptography} -In \emph{symmetric cryptography}, a single key is used to cipher and decipher the message ($K = K^{-1}$). Figure \ref{fig:sc} illustrates communication between Alice and Bob. +\subsection{Symmetric Cryptography} +In \emph{symmetric cryptography}, a single key is used to cipher and decipher the message ($K = K^{-1}$). Figure \ref{fig:sc} illustrates communication between Alice and Bob. \begin{figure}[h] - \centering - \includegraphics[scale=0.4]{figures/lab2_sc.pdf} - \caption{Symmetric cryptography. Alice sends a message to Bob. $K = K^{-1}$} - \label{fig:sc} +\centering +\includegraphics[scale=0.4]{figures/lab2_sc.pdf} +\caption{Symmetric cryptography. Alice sends a message to Bob. $K = K^{-1}$} +\label{fig:sc} \end{figure} -Symmetric cryptography is considered the fastest in terms of encryption and decryption speed, however it does not come without its drawbacks. -The first one is \textbf{sharing the key}. In order to communicate with each other, they must first know the shared secret (i.e the key) beforehand. This isn't a trivial problem, since if the two entities have never talked with each other (or might not even know who is the other entity), they may not have a way to securely share the key. +There are some problems in practice: the party generating the key must deliver it in a confidential way. The party receiving the key must assure the key received is the key intended to be sent, and it should be able to confirm it is the most recent. Moreover, there needs to be a key, for each pair of parties. -The other one is \textbf{the number of keys} used to communicate between entities. We need one key to safely between 2 entities, however if we want a system with $N$ entities all wanting to communication with each other, we will need $N^2$ keys, which is not scalable to bigger environments. +Typical examples of symmetric cryptography include the TEA algorithm, the Data Encryption Standard (DES), Triple DES, Twofish, and the AES. -Typical examples of symmetric cryptography include the TEA algorithm, the Data Encription Standard (DES), Triple DES, Twofish, Diffie Hellman and the AES. +\subsection{Asymmetric Cryptography} +In \emph{asymmetric cryptography}, there are two keys: one is used for cipher, and another to decipher. Normally, we call \emph{public key} to the key that ciphers ($K_u$), and \emph{private key} to the key that deciphers ($K_r$). We remark that it is also possible to cipher with the private key and decipher with the public key. Figure \ref{fig:pc} illustrates this process. +Contrarily to symmetric key cryptography, there is no need to assure the confidentiality of the public key. The party that wants to communicate needs to publish the public key (assuring its integrity and authenticity). There is no need to distribute a pair of keys per pair of parties. -\subsubsection{Asymmetric Cryptography} -In order to solve the problem of the number of keys, we can use \emph{asymmetric cryptography}. -In \emph{asymmetric cryptography}, there are two keys: a \emph{public key} and a \emph{private key}. As the name implies, the \emph{public key} is meant to be public and out in the open, while the \emph{private key} should be kept by an entity and \textbf{never be shared}. The nice property about these keys is that messages encrypted with the public key can \textbf{only be decrypted by the private key}. The same goes for the private key, with messages encrypted with it only being decrypted with the public key. - -This give us a great advantage with key sharing. Since only we have our own private key, only we can decrypt any message encrypted with the public, which anyone can have. This brings down to only needing 2 keys per entity to communicate. So in a system with $N$ entities we only need $2N$ keys. +Many ciphers, such as \emph{RSA}, are not considered \emph{totally safe}, as an attacker can discover the key. However, RSA is considered \emph{practically safe}, given that there is a need for a substantial computational investment to find the key. \begin{figure}[h] - \centering - \includegraphics[scale=0.4]{figures/lab2_pc.pdf} - \caption{Asymmetric cryptography. Alice obtains Bob's public key, and uses it to encrypt and send a message.} - \label{fig:pc} +\centering +\includegraphics[scale=0.4]{figures/lab2_pc.pdf} +\caption{Asymmetric cryptography. Alice obtains Bob's public key, and uses it to encrypt and send a message.} +\label{fig:pc} \end{figure} -Using the private key to encrypt content also give us some nice properties, which we will talk later in this lab. - -Apart from the key sharing advantage, it is much computational heavier to encrypt and decrypt messages, which may be undesirable if we want fast communication. - -Many ciphers, such as \emph{RSA}, are not considered \emph{totally safe}, as it is possible for an attacker to discover the key. However, RSA is considered \emph{practically safe}, given that there is the need of a very large computational investment to find the key. -Nevertheless, with this in mind, it is good practices to avoid overusing the private/public key pair. - -Typical examples of asymmetric cryptography include elliptic curve algorithms and RSA. - -\subsection{Authentication} - -Although these encryption methods can be used for confidentiality, it is important to note that \textbf{it is not their only purpose}. With the right system and preparation, they can also be used to provide \textbf{entity authentication}. - -\emph{Authentication} aims to provide an answer to the question, ``who is the client?", or the subject that issues an access control request. It is concerned to identifying the subject that asks permission to realize a certain action on a certain object, in a specific context. -This is needed to give \emph{Authorization} to perform a certain action. - -In order to assure authentication, three properties must be present in a message: -\begin{itemize} - \item Identification; - \item A proof of identity; - \item Freshness; - \item Integrity. -\end{itemize} - -We will see how each one is important for authentication. - -\subsubsection{Identification} - -Identification is rather trivial. When sending a message to someone, we first need to identify ourselves to be able to be verified. Without it, we would not be able to distinguish between entities. - -\subsubsection{Proof of Identity} - -In order to authenticate ourselves, we need some form of \textbf{proof} that we are who we say we are. -This can be done in one of three ways: -\begin{enumerate} - \item Something we \emph{know}; - \item Something we \emph{are}; - \item Something we \emph{have}; -\end{enumerate} - -\emph{Something we know} refers to things like passwords, which is information that only we as entities know and (supposedly) nobody else does. - -\emph{Something we are} refers to characteristics that are unique to us, specifically bio-metric information. This commonly includes eye scanners, fingerprint scanners and our face. - -\emph{Something we have} refers to something like a physical badge, a smartphone or a \emph{cryptographic key}. -Given the assumption that when a symmetric key was created it was shared correctly and safely, we can trust that someone that encrypts with that key is in fact authenticated and authorized to send messages. - -In asymmetric encryption, this can only be assured by encrypting with the \emph{private key}, since the public key is public to all the network. - -However, encrypting with the private key gives us a nice property that is not present with symmetric keys, which is non-repudiation. +Typical examples of symmetric cryptography include elliptic curve algorithms, Diffie Hellman, and RSA. -\emph{Non-Repudiation} means that the message authorship cannot be disputed. With symmetric encryption, both entities have the same key and as such, from a third party view we wouldn't be able to distinguish which entity sent the message. With asymmetric encryption however, only one entity has the private key. As such, any message signed with it can only have come from that entity. +\subsection{Digital Signatures} +The purpose of digital signatures is to assure \emph{integrity, authenticity, and non-repudiation} of information (e.g., messages, documents, transactions). Integrity regards preventing changes from the document. Authenticity allows us to uniquely identify the subject that signed the document. Non-repudiation enforces accountability for the content created. In other words, the entity signing a document cannot deny the signature. -\subsubsection{Freshness} - -We now have a way to prove our identity, however if we would send the same message with our proof over and over, a malicious user could copy our message and use it to authenticate himself as being us. This is called a \emph{replay attack}. - -This is possible because we have no way to verify the \emph{freshness} of the message, meaning how recent it is. - -We can add freshness in multiple ways: - -\begin{itemize} - \item \emph{Nonce}. Meaning \textbf{number only used once}, we can use it to avoid replay attacks by rejecting messages with a nonce that we have previously seen. The most common case of a nonce is a counter, which we increment every time we send a new message. - \item \emph{Timestamp}. There is not better way to verify if a message was recently sent by adding the timestamp with which was sent. This way, we only accept messages sent in the last $X$ minutes. Although it may look trivial to use, timestamps are discouraged, since it requires the clocks between nodes to be synchronized. - \item \emph{Challenge-Response}. When we want to authenticate, we first ask the entity with which we want to authenticate for a challenge, which can be anything from a number to a string. In order to show freshness, we must alter that challenge in some way, which will give us the response that we will send to the destiny entity. If verified correctly, we are authenticated. It is of note that this in itself can be considered an authentication technique, since answering the challenge is "something we know", although it is not very safe. -\end{itemize} - -\subsubsection{Integrity} - -So we now have a way to prove our identity and can differentiate between old and new messages. However, we are still missing a piece. What stops an attacker from copying an authentication message, altering the Nonce to an unused value and sending that altered message? - -We need a way to verify that the message itself was not altered during transmission, i.e to verify its \emph{Integrity}. We need something that is dependent on the content of the message such that if the message was altered in any way we are able to verify it. - -To this purpose, we use hash functions \cite{rogaway2004}. A hash function receives a text and returns a sequence of bits of fixed length such that it follows three properties: +A signature of a piece of information (also called digest) is made through a hash function \cite{rogaway2004}. Typically, hash functions are very efficient. A hash function receives a text and returns a sequence of bits of fixed length such that it follows three properties: \theoremstyle{definition} \begin{definition}{Pre-image resistance:} -It is computationally infeasible to find any input $M$ that hashes to a given output $D$. In other words, given $D$, it is computationally infeasible to find $M$ such that $H(M) = D$. +It is computationally infeasible to find any input $M$ that hashes to a given output $D$. In other words, given $M$, it is computationally infeasible to find $M$ such that $H(M) = D$. \end{definition} @@ -277,131 +180,86 @@ \subsubsection{Integrity} \end{definition} - -This corresponds to saying that it is difficult to find a different input that produces the same hash. + +This corresponds to saying that it is difficult to find a different input that produces the same hash. \theoremstyle{definition} \begin{definition}{Collision Resistance:} It is computationally infeasible to find any two distinct inputs $M$, $M'$, that hash to the same output $D$, i.e., such that $H(M) = H(M') = D$. - - -\end{definition} - -Collision resistance implies second pre-image resistance, but does not guarantee pre-image resistance. Non invertible hash functions aim to create a unique digest. It is then desirable that hash functions are collision resistance, otherwise an attacker could try to find a different input that matches the same digest. - -Famous examples are the MD5 hash function (deprecated) and SHA family of functions (SHA256, SHA512) \cite{conrad2016}. -It is important to notice that if we insert a digest in a message unprotected, it gives us \textbf{no integrity}, because nothing stops an attacker from altering the message and recalculating the hash of the now altered message. As such, for it to be useful, it must be \emph{encrypted} so it cannot be altered. -We now have all the ingredients to make an authentication message. Using these building blocks, a message should look like this: - -$Message = (M, E_k(Digest), Freshness)$ +\end{definition} -Where: -\begin{itemize} - \item $M$ = Message to be sent; - \item $E_k$ = Key with which to authenticate; - \item $Digest$ = H(M + Freshness); - \item $Freshness$ = Any form of previously mentioned freshness methods. -\end{itemize} +Collision resistance implies second pre-image resistance but does not guarantee pre-image resistance. Noninvertible hash functions aim to create a unique digest. It is then desirable that hash functions are collision resistance. Otherwise an attacker could try to find a different input that matches the same digest. Famous examples are the MD5 hash function (deprecated), MAC, HMAC, SHA256 \cite{conrad2016}. -\subsubsection{Digital Signatures} -When using a private key to sign the digest, we call this a \emph{Digital Signature}. Digital signatures are useful when we want to assure \emph{integrity, authenticity, and non-repudiation} of information (e.g., messages, documents, transactions). \begin{figure}[h] - \centering - \includegraphics[scale=0.4]{figures/digital_signature_create.pdf} - \caption{Simple digital signature process} - \label{fig:ds_create} +\centering +\includegraphics[scale=0.4]{figures/digital_signature_create.pdf} +\caption{Simple digital signature process} +\label{fig:ds_create} \end{figure} -Figure \ref{fig:ds_create} depicts a simplified process underlying the creation of a secure message, ready to be transmitted. Alice creates a document, and calculates its digest using a hash function. After that, she encrypts the digest using a cipher. Finally, the document to be sent consists in the original message, plus an encrypted digest. The validation process is shown by Figure \ref{fig:ds_validate}: Bob retrieves the combination message and encrypted hash of the message. With the public key of the sender, Bob decrypts the digest of the message: created by Alice (authenticity and non-repudiation). After that, Bob recalculates the digest from M, and compares it with the sent digest. If they are the same, it means that the information did not change - and hence, we assure integrity. Note that this process does not include Freshness, as such it is vulnerable to \emph{replay attacks}. +Figure \ref{fig:ds_create} depicts a simplified process underlying the creation of a secure message, ready to be transmitted. Alice creates a document and calculates its digest using a hash function. After that, she encrypts the digest using a cipher—finally, the document to be sent consists of the original message and an encrypted digest. The validation process is shown by Figure \ref{fig:ds_validate}: Bob retrieves the combination message and encrypted hash of the message. Alice, Bob decrypts the digest of the message with the public key of the sender, created by Alice (authenticity and non-repudiation). After that, Bob recalculates the digest from M and compares it with the sent digest. If they are the same, it means that the information did not change - and hence, we assure integrity. \begin{figure}[h] - \centering - \includegraphics[scale=0.4]{figures/digital_signature_validate.pdf} - \caption{Simple digital signature validation process} - \label{fig:ds_validate} +\centering +\includegraphics[scale=0.4]{figures/digital_signature_validate.pdf} +\caption{Simple digital signature process} +\label{fig:ds_validate} \end{figure} -There are other well established authentication protocols that the reader should be aware that do not follow this kind of process, specially for symmetric encryption, like the Message Authentication Code (MAC) family, which includes MAC, HMAC, CBC-MAC and many others. +However, this solution may not work in all cases (see Exercise 2). +\subsection{ Authentication, authorization, and accounting} +\emph{Authentication} aims to provide an answer to the question, ``who is the client?", or the subject that issues an access control request. It is concerned to identify the subject that asks permission to realize a particular action on a certain object in a specific context. \emph{Authorization} provides an answer to, ``what is the client allowed to do?". The \emph{Accounting}, on its turn, answers to the question, "what did the client do?". These fundamental security building blocks enable networks to have effective and dynamic security. -%Future work -% Add more schemes and images -% Session keys -% Perfect Forward Secrecy -% Authenticated Diffie-Helman -% Certificates +Authentication can be performed in several ways: \begin{itemize} +\item \textit{Shared-key}: A username is associated with a password. +\item \textit{One-time password}: a token is used to generate an access code valid for one access only. This process has to be synced with a token server (PIP). +\item \textit{Digital certificate}: emitted and signed by an authority. This certificate contains information about the entity it refers to. It contains a private-public key pair, used for encrypting and decrypting the same message. +\item \textit{Biometric credential}: based on what the client is, such as a fingerprint. +\end{itemize} + +The authorization phase takes place after authentication. It verifies if a subject can perform an action on an object, based on access control policies. Accountability tracks all actions realized over a system: every access control request and corresponding decision. This allows to associated performed actions on objects to subjects. -\comment{ \subsection{Asymetric Key Cryptography: RSA} -RSA (Rivest–Shamir–Adleman) is one of the first public-key algorithms, used for secure data transmission. RSA ciphers per blocks of dimension $k$, such that $2^k< N$. +RSA (Rivest–Shamir–Adleman) is one of the first public-key algorithms used for secure data transmission. RSA ciphers per block of dimension $k$, such that $2^k< N$. One can divide RSA's excution into several steps: \begin{enumerate} - \item Key Generation - \item Encryption - \item Decryption +\item Key Generation +\item Encryption +\item Decryption \end{enumerate} The first step of RSA consists in key generation. Two keys are considered, the public key $K_u = (N,e)$ and the private key, $K_r = (N,d)$. We choose two prime numbers $p$ and $q$, such that $N = p.q$, and $Z = (p - 1).(q - 1)$. More concretely: \begin{itemize} - \item Choose primes $p$ and $q$, and $N = p.q$ - \item Choose $e$ such that the greatest common divisor between $e$ and $(p-1).(q-1)$ is 1. - \item Calculate $d$ such that $e.d \mod (p - 1).(q - 1) = 1$. Restriction: $0 < d < (p - 1).(q - 1)$. - \item Communicate the public key $(N,e)$, and the private key $(N,d)$. +\item Choose primes $p$ and $q$, and $N = p.q$ +\item Choose $e$ such that the greatest common divisor between $e$ and $(p-1).(q-1)$ is 1. +\item Calculate $d$ such that $e.d \mod (p - 1).(q - 1) = 1$. Restriction: $0 < d < (p - 1).(q - 1)$. +\item Communicate the public key $(N,e)$, and the private key $(N,d)$. \end{itemize} After keys are generated, text can be ciphered following a specific enconding, e.g., ASCII\footnote{https://www.ascii-code.com/} encoding, according to the following Figure \ref{fig:ascii}: \begin{figure}[h] - \centering - \includegraphics[scale=0.2]{figures/ascii.png} - \caption{ASCII encoding} - \label{fig:ascii} +\centering +\includegraphics[scale=0.2]{figures/ascii.png} +\caption{ASCII encoding} +\label{fig:ascii} \end{figure} Therefore, number 65 represents character `A'. The cryptogram can be calculated as follows: $C = M^e \mod N$. For decryption, a similar operation is performed, with the private key: $M = C^d \mod N$. -In practise, to encode a large number, one can define $B <= N$, such that the message to be encrypted is divided into blocks of $B$ digits each one. The cryptogram is sent into blocks. The decryption phase is analogous: it decrypts the several blocks received into a single message. +In practice, to encode a large number, one can define $B <= N$, such that the message to be encrypted is divided into blocks of $B$ digits each one. The cryptogram is sent into blocks. The decryption phase is analogous: it decrypts the several blocks received into a single message. -For the attacker to discover $K_r$, he needs to know $Z$. To know $Z$, the attacker needs to discover $p$ and $q$. As $p$ and $q$ are prime numbers, it is computationally expensive to discover both, such that $p.q = n$. At the key generation step, one should destroy $p$ and $q$, so the public and private keys cannot be recalculated. Although RSA can be considered safe, it is important for the exponents to be very large numbers; if not, attacks are possible \cite{wiener90}. -} +For the attacker to discover $K_r$, he needs to know $Z$. To know $Z$, the attacker needs to discover $p$ and $q$. As $p$ and $q$ are prime numbers, it is computationally expensive to discover both, such that $p.q = n$. At the key generation step, one should destroy $p$ and $q$, so the public and private keys cannot be recalculated. Although RSA can be considered safe, the exponents need to be very large numbers; if not, attacks are possible \cite{wiener90}. -%Future work -%\subsubsection{Digital Signatures: ECDSA algorithm} -%ECDSA stands for “Elliptic Curve Digital Signature Algorithm” - a popular digital signature algorithm. -%To know more about ECDSA please consult the sources: \cite{ecdsa1,ecdsa2} \section{Hands on Crypto} -\subsection*{Confidentiality} -\subsubsection*{Exercise 1: When should we use symmetric encryption? What about asymmetric?} - -\subsection*{Authentication} -\subsubsection*{Exercise 2: Are we able to get Confidentiality without Authentication? What about the other way around?} - -\subsubsection*{Exercise 3: Why would we ever want Authentication without Confidentiality? What is the problem with encrypting the whole message?} - -\subsubsection*{Exercise 4: Why is encrypting the hash for authentication enough for attackers to be unable to alter it?} - -\subsubsection*{Exercise 5: Is it safe to not encrypt the freshness of a message? Can an attacker exploit anything by knowing the freshness? } - -\subsubsection*{Exercise 6: What are the disadvantages of using each type of proof of identity? (Something we \emph{know, are and have})} - -\subsubsection*{Exercise 7: Nowadays a lot of services are using Multi-factor authentication (meaning more than one way of authentication). What is the advantage of it?} - -\subsection*{Freshness} -\subsubsection*{Exercise 8: What is the possible disadvantage of using Nonces?} - -\subsubsection*{Exercise 9: How could we minimize this disadvantage?} - -\subsubsection*{Exercise 10: Give an example of what could be considered a challenge-response} - -\comment{ - -\subsubsection*{Exercise 1: How many combinations can the MD5 and SHA256-3 algorithms generate? What is the likelihood of a hash collision with MD5? And with SHA256-3? Is SHA256-3 safer than MD5?} +\subsubsection*{Exercise 1: How many combinations can the MD5, and SHA256-3 algorithms generate? What is the likelihood of a hash collision with MD5? And with SHA256-3? Is SHA256-3 safer than MD5?} \subsubsection*{Exercise 2: Refer to Figures \ref{fig:ds_create} and \ref{fig:ds_validate}. Is this approach of signing and validating a document secure?} @@ -412,7 +270,7 @@ \subsubsection*{Exercise 3: Fork the course repository \footnote{https://github. \subsubsection*{Exercise 4: Calculate the RSA keys associated with primes $p = 7$ and $q = 29$} -Hint: you may use the support code to create a test case that helps you answering the following exercises. +Hint: you may use the support code to create a test case that helps you answer the following exercises. \subsubsection*{Exercise 5: Given that the public key $K_u = (N,e) = (143, 7)$ and the private key is $K_r = (N,d) = (143, 103)$, encrypt the following message: ``P''} @@ -420,8 +278,10 @@ \subsubsection*{Exercise 5: Given that the public key $K_u = (N,e) = (143, 7)$ a \subsubsection*{Exercise 6: Decrypt the criptogram 80, given that $p = 3$ and $q = 31$ } -%\subsubsection*{Exercise 7: Adapt the code to support text encrpytion in the ASCII format, and separation into different blocks} -} +%\subsubsection*{Exercise 7: Adapt the code to support text encryption in the ASCII format, and separation into different blocks} + + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT ENDS HERE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% diff --git a/source/Lab02/Instructors Guide/assignment.tex b/source/Lab02/Instructors Guide/assignment.tex index f9cdc61..ffdad4f 100644 --- a/source/Lab02/Instructors Guide/assignment.tex +++ b/source/Lab02/Instructors Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -19,8 +19,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -49,7 +49,7 @@ commentstyle=\color{green}\itshape, moredelim=[s][\color{blue}\bfseries]{C:}{\>} } -\newcommand{\comment}[1]{} + \lstdefinestyle{Bash} {% language=bash, breaklines=true, @@ -70,16 +70,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 2\\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & 21 Sept 2020 \\ +\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Background: Cryptography \& Security} & \textbf{Due Date:} & 28 Sept 2020\\ +\multicolumn{2}{|l|}{Background: Cryptography \& Security} & \textbf{Due Date:} & \\ \hline \end{tabularx}} \rhead{} @@ -94,77 +94,24 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 \section*{Instructors Guide} -This document provides a proposal of solution for LAB\#02, which concerns the an introduction to crypography and security. The questions made have the goal to amke the students consider and reach their own conclusions about the advantages, disadvantages and possible vulnerabilities of the various cryptographic properties. +This document provides a proposal for a solution for LAB\#02, which concerns an introduction to cryptography and security. In particular, the RSA algorithm is introduced. Most of the solutions can be found at \cite{conrad2016,rogaway2004,md2020}. Note: You can also experiment with Hyperledger Ursa, a crypto library \footnote{https://github.com/hyperledger/ursa}, which supports symmetric encryption and digital signatures. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT STARTS HERE -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\subsection*{Confidentiality} -\subsubsection*{Exercise 1: When should we use symmetric encryption? What about asymmetric?} - -A: We want to use symmetric encryption when we have a secure way to establish keys with the target entity, with a short communication lifespan and when we communicate with a small number of entities. -We want to use asymmetric encryption when we dont have a secure platform to share/generate keys and will communicate with a wide number of people. - -\subsection*{Authentication} -\subsubsection*{Exercise 2: Are we able to get Confidentiality without Authentication? What about the other way around?} - -A: Yes. When using asymmetric encryption, encrypting with the public key will give confidentiality but not authentication of the user. With regard to symmetric encryption, as we have seen previously in the lab, encrypting with a key is a step but not enough to guarantee secure authentication. - -For the other way around, we can also get authentication without confidentiality. For secure authentication, we only need the digest of the message to be encrypted so it cannot be altered, keeping the rest of the message in plain text. - -\subsubsection*{Exercise 3: Why would we ever want Authentication without Confidentiality? What is the problem with encrypting the whole message?} - -A: Depending on the use case, it may be undesirable to encrypt the whole message, since the longer the message the more computationally heavy it is to decrypt it, specially when using asymmetric encryption. As such, if the information does not require to be confidential (like a post to a forum that would be public nevertheless), it is best to only \textbf{encrypt the necessary}. - -\subsubsection*{Exercise 4: Why is encrypting the hash for authentication enough for attackers to be unable to alter it?} - -A: If an attacker wants to alter the contents of a message, they need to change the hash as well. However, in order to change the hash, they need to know the key used to encrypt. Since one of the assumptions is that the keys are kept secret and only the authorized people have them, an attacker is unable to alter it. They CAN however use a past encrypted digest to their advantage, so if the system is not robust it may be a vulnerability. - -\subsubsection*{Exercise 5: Is it safe to not encrypt the freshness of a message? Can an attacker exploit anything by knowing the freshness? } - -A: It depends on the robustness of the system. An attacker could possibly use the answers to challenge-response or nonce to their advantage and if not robustness is added, even re-utilize those answers. Nevertheless, assuming a robust system is implement, it should not be a problem. - -\subsubsection*{Exercise 6: What are the disadvantages of using each type of proof of identity? (Something we \emph{know, are and have})} - -A: -\begin{itemize} - \item Something we are (Biometric): It requires specialised hardware (although nowadays things like fingerprint sensors are getting much more common), we cannot give to another person (the same way we could give our login information to a friend) and it is not something we can change (i.e. if someone obtains a copy of our fingerprint we cannot alter it like we would be able to change a password). NOTE - Also fine tuning between false positives and false negatives, but we do not talk about it in the lab. - \item Something we know (passwords): It requires the entity to memorize the passwords used for each system and if weak are prone to brute force attacks. - \item Something we have (Phsyical and cryptographic keys): With physical we need to carry it around everywhere, which can be a burden if a lot of systems use it. With cryptographic keys they get weaker the more we use it, so they need to be renewed from time to time. -\end{itemize} - -\subsubsection*{Exercise 7: Nowadays a lot of services are using Multi-factor authentication (meaning more than one way of authentication). What is the advantage of it?} - -A: Being the use of passwords or pins the most common way of authentication, it has become weaker with the advancements of brute-force and dictionary attacks. As such, by combining multiple methods of authentication, even if a password was stolen or leaked they would still lack a piece to authenticate as ourselves. - -\subsection*{Freshness} -\subsubsection*{Exercise 8: What is the possible disadvantage of using Nonces?} - -A: In order to use nonces, the target entity must remember all the nonce values it has seen from previous messages from all entities it knows. This can become a big burden in terms of memory consumption. - -\subsubsection*{Exercise 9: How could we minimize this disadvantage?} - -A: By adding timestamps as well as nonces, we can do a system in which each Nonce is kept for a certain time period. This way, we can garbage collect previous Nonces that are outside of the time window and attackers wouldnt be able to replay attack because the timestamp associated with the message would be outside of the maximum time window allowed. - -\subsubsection*{Exercise 10: Give an example of what could be considered a challenge-response} - -A: For example, being given a number as a challenge and having to return the next one as response. - -\comment{ - \section{RSA} \label{sec:rsa} -\subsubsection*{Exercise 1: How many combinations can the MD5 and SHA256-3 algorithms generate? What is the likelihood of a hash collision with MD5? And with SHA256-3? Is SHA256-3 safer than MD5?} +\subsubsection*{Exercise 1: How many combinations can the MD5, and SHA256-3 algorithms generate? What is the likelihood of a hash collision with MD5? And with SHA256-3? Is SHA256-3 safer than MD5?} MD5 is a 128-bit cryptographic hash function, which means that digests are distributed over a 128-bit space. MD5 can output $2^{128}$ digests. Two files should have a $\frac{1}{2^{128}}$ change of collision. However, due to the birthday paradox, that probability would be $\frac{1}{2^{64}}$. -SHA256-3 is a 256-bit cryptographic hash function, which means that digests are distributed over a 256-bit space. SHA256-3 can output $2^{256}$ different digests. +SHA256-3 is a 256-bit cryptographic hash function, which means that digests are distributed over a 256-bit space. SHA256-3 can output $2^{256}$ different digests. Two files should have a $\frac{1}{2^{256}}$ change of collision. However, due to the birthday paradox, that probability would be $\frac{1}{2^{128}}$. MD5 is considered to be vulnerable to collision attacks \cite{wang2005}. @@ -172,14 +119,14 @@ \subsubsection*{Exercise 1: How many combinations can the MD5 and SHA256-3 algor \subsubsection*{Exercise 2: Refer to Figures 3 and 4. Is this approach of signing and validating a document secure?} -This approach is generally considered secure, given that Alice's public key is authentic (to prevent man-in-the-middle attacks), and that the hash function that creates the digest is secure. The key distribution should be done in a secure way. See public-key infrastructure, public key certificates, and the X.509 norm. +This approach is generally considered secure, given that Alice's public key is authentic (to prevent man-in-the-middle attacks). The hash function that creates the digest is secure. The key distribution should be done securely. See public-key infrastructure, public key certificates, and the X.509 norm. \subsubsection*{Exercise 4: Calculate the RSA keys associated with primes $p = 7$ and $q = 29$} Public key is $K_u = (N,e)$ and the private key, $K_r = (N,d)$. $N = p.q = 7.29 = 203$ $Z = (p - 1).(q - 1) = 6.28 = 168$ -$e$ is chosen such that the greatest common divisor between e and 168 is be one. +$e$ is chosen such that the greatest common divisor between e and 168 is be one. $e = 5, 11, 13, 17, 19, ... 157, 163, 167$. $d = 101$, because 101 is the only number such that $e.d \mod (p - 1).(q - 1) = 1$. Restriction: $0 < d < (p - 1).(q - 1)$. The public key $K_u = (N,e)$ is $(203, 5)$ and the private key, $K_r = (N,d)$ is $(203,101)$. @@ -194,8 +141,8 @@ \subsubsection*{Exercise 6: Decrypt the criptogram 80, given that $p = 3$ and $q The private key is, therefore, $K_r = (N,d) = (93,43)$. $M = C^d \mod n = 80^43 mod 93 = 87$. -%\subsubsection*{Exercise 7: Adapt the code to support text encrpytion in the ASCII format, and separation into different blocks} -} +%\subsubsection*{Exercise 7: Adapt the code to support text encryption in the ASCII format, and separation into different blocks} + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT ENDS HERE diff --git a/source/Lab03/Guide/assignment.tex b/source/Lab03/Guide/assignment.tex index 2673c90..b31fd07 100644 --- a/source/Lab03/Guide/assignment.tex +++ b/source/Lab03/Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -24,8 +24,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -75,16 +75,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 3 \\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & - \\ +\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & - \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Background: A Primer on Blockchain Technology} & \textbf{Due Date:} & - \\ +\multicolumn{2}{|l|}{Background: A Primer on Blockchain Technology} & \textbf{Due Date:} & - \\ \hline \end{tabularx}} \rhead{}} @@ -99,7 +99,7 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 \section*{Preliminary Notes} -This class provides background about blockchain technology. This laboratory is based on previous work elaborated by the contributors \cite{belchior2019_audits,belchior2020,belchior2019_thesis}. We recommends students to have attended Lab1 and Lab2. +This class provides background about blockchain technology. This laboratory is based on previous work elaborated by the contributors \cite{belchior2019_audits,belchior2020,belchior2019_thesis}. We recommend students have attended Lab1 and Lab2. %The reference section should be viewed as a ``additional readings reference'' - if you would like more information about the topic. @@ -107,43 +107,43 @@ \section*{Preliminary Notes} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT STARTS HERE -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{A Primer on Blockchain} -We now introduce Blockchain, whereby its building blocks are rooted on Distributed Systems and Cryptography (see Lab01 and Lab02). +We now introduce Blockchain, whereby its building blocks are rooted in Distributed Systems and Cryptography (see Lab01 and Lab02). %TODO motivation, use cases %\subsection{What is Blockchain For} \subsection{A Technical Viewpoint} -The term blockchain is generally referred to as a replicated data structure: a persistent, auditable, immutable, append-only distributed ledger. The blockchain is maintained by a group of physical machines, nodes (or peers or participants). Nodes are not trusted individually to maintain the distributed ledger; they are trusted as a group, due to their number and diversity. As long as nodes have conflicting interests, collusion between them will probably not happen: and thus the network can be considered safe. Blockchain, in fact, implements many concepts discussed in Lab01: for instance, it is a byzantine fault tolerant (or just crash fault tolerant) replicated state machine that agrees on a global state via consensus. The blockchain is secure due to a combination of decentralization and cryptograpy: transactions are aggregated in blocks, linked by the cryptographic hash of the previous block. If a participant changes a prior block, its hash will change, as well as the posterior hashes. +The term blockchain is generally referred to as a replicated data structure: a persistent, auditable, immutable, append-only distributed ledger. The Blockchain is maintained by a group of physical machines, nodes (or peers or participants). Nodes are not trusted individually to maintain the distributed ledger; they are trusted as a group due to their number and diversity. As long as nodes have conflicting interests, collusion between them will probably not happen: and thus, the network can be considered safe. Blockchain, in fact, implements many concepts discussed in Lab01. For instance, a byzantine fault-tolerant (or just crash fault-tolerant) replicated state machine agrees on a global state via consensus. The Blockchain is secure due to a combination of decentralization and cryptography: transactions are aggregated in blocks, linked by the cryptographic hash of the previous block. If a participant changes a previous block, its hash will change and the posterior hashes. -Most blockchains serve as a runtime for programs called smart contracts. The execution of smart contracts is originated by transactions. Smart contracts are executed in a virtual machine, e.g., in the Ethereum Virtual Machine (EVM) in Ethereum, one of the most used blockchains. Smart contracts can abstract the representation of currency, resources, assets, access, equity, identity, collectibles, via tokens. +Most blockchains serve as a runtime for programs called smart contracts. The execution of smart contracts is originated by transactions. Smart contracts are executed in a virtual machine, e.g., in the Ethereum Virtual Machine (EVM) in Ethereum, one of the most used blockchains. Smart contracts can abstract the representation of currency, resources, assets, access, equity, identity, collectibles via tokens. There are different types of blockchains, which can typically be categorized into permissionless blockchains and permissioned blockchains. \subsubsection{Permissionless blockchains} -Public blockchains can be accessed and utilized by anyone with Internet access. Permissionless blockchains do not require participants to be identified to participate. We use public and permissionless as synonyms. Typically, in such networks, the participants are rewarded for supporting the ecosystem, making it self-sustainable. The first entirely decentralized digital currency, Bitcoin, is a permissionless peer-to-peer version of electronic cash, which would allow transactions to go directly from one party to the other, excluding the need for any intermediary. Bitcoin quickly gained popularity because it solved the famous double-spending problem, without the need of any central authority. +Public blockchains can be accessed and utilized by anyone with Internet access. Permissionless blockchains do not require participants to be identified to participate. We use public and permissionless as synonyms. Typically, in such networks, the participants are rewarded for supporting the ecosystem, making it self-sustainable. The first entirely decentralized digital currency, Bitcoin, is a permissionless peer-to-peer version of electronic cash, which would allow transactions to go directly from one party to the other, excluding the need for any intermediary. Bitcoin quickly gained popularity because it solved the famous double-spending problem without the need of any central authority. Examples: Bitcoin, Ethereum, EOS, Ripple, IOTA. \subsubsection{Permissioned blockchains} -Permissioned blockchains identify and associate participation right to participants. Private blockchains require credentials for participants to vizualize it. We use private and permissioned as synonyms. Private blockchains are the property of an organization or consortium of organizations. If more than one organization is included in the management of the blockchain, it is called a community or federated blockchain. A permissioned solution seems suitable for companies aiming for competitiveness that blockchain technology can offer, while protecting sensitive information. Write permissions are kept centralized to specific nodes within the organization. Therefore, a trade-off between control and decentralization exists. On a technological point of view, compared to public blockchains, there is an aggravated risk of subversion, as there are fewer participants in the network that validates transactions. Neverthesless, in practice, as the network only allows identified participants, it is easier to identify a culprit. +Permissioned blockchains identify and associate participation right to participants. Private blockchains require credentials for participants to vizualize it. We use private and permissioned as synonyms. Private blockchains are the property of an organization or consortium of organizations. Suppose more than one organization is included in the management of the Blockchain. In that case, it is called a community or federated Blockchain. A permissioned solution seems suitable for companies aiming for competitiveness that blockchain technology can offer while protecting sensitive information. Write permissions are kept centralized to specific nodes within the organization. Therefore, a trade-off between control and decentralization exists. From a technological point of view, there is an aggravated risk of subversion compared to public blockchains. There are fewer participants in the network that validates transactions. Nevertheless, in practice, as the network only allows identified participants, it is easier to identify a culprit. Examples: Hyperledger Fabric, Hyperledger Indy, Hyperledger Burrow, Hyperledger Sawtooth, R3 Corda, and Multichain. \subsection{Comparison} -Public and private blockchains are essencially different in nature. Inside each category, blockchains are also very different: their consensus, networking, data, and application layers can differ substantially, leading to a wide-range of choices. This leads to fragmentation, where some blockchains are immature (not stable). While this can be a big problem, research in blockchain interoperability attenuates this \cite{belchior2020}. +Public and private blockchains are essentially different in nature. Inside each category, blockchains are also very different: their consensus, networking, data, and application layers can differ substantially, leading to a wide range of choices. This leads to fragmentation, where some blockchains are immature (not stable). While this can be a big problem, research in blockchain interoperability attenuates this \cite{belchior2020}. \begin{figure}[H] - \centering - \includegraphics[scale=0.45]{figures/PublicVsPrivate.png} - \caption{Comparison between public and private blockchain} - \label{fig:bcs} +\centering +\includegraphics[scale=0.45]{figures/PublicVsPrivate.png} +\caption{Comparison between public and private blockchain} +\label{fig:bcs} \end{figure} % Source of picture: https://101blockchains.com/private-blockchain-vs-consortium-blockchain/#prettyPhoto @@ -151,39 +151,24 @@ \subsection{Comparison} Figure \ref{fig:bcs} depicts two blockchains: Hyperledger Fabric, a permissioned blockchain; and Bitcoin, a permissionless -In Hyperledger Fabric, the consensus is based on endorsement policies. In Fabric, a client (C) sends a transaction proposal to the peer nodes (P), and obtains a signed transaction, called an endorsement (steps 1 and 2). An orderer validates the endorsements and builds a block with valid transactions, appending it to the ledger (steps 3 and 4). In Bitcoin, the consensus is based on the notion of Proof-of-Work (PoW), a cryptographic puzzle that mining nodes need to solve in order to build a valid block. This corresponds roughly to Fabric’s steps 1-3. After a node finds a solution to PoW, it then can propose a block of transactions to be appended to the ledger (step 4). +In Hyperledger Fabric, the consensus is based on endorsement policies. In Fabric, a client (C) sends a transaction proposal to the peer nodes (P), and obtains a signed transaction, called an endorsement (steps 1 and 2). An orderer validates the endorsements and builds a block with valid transactions, appending it to the ledger (steps 3 and 4). In Bitcoin, the consensus is based on the notion of Proof-of-Work (PoW), a cryptographic puzzle that mining nodes need to solve to build a valid block. This corresponds roughly to Fabric’s steps 1-3. After a node finds a solution to PoW, it then can propose a block of transactions to be appended to the ledger (step 4). \begin{figure} - \centering - \includegraphics[scale=0.4]{figures/Fig1.pdf} - \caption{High-level comparison of Hyperledger Fabric and Bitcoin, from \cite{belchior2020survey}} - \label{fig:bcs} +\centering +\includegraphics[scale=0.4]{figures/Fig1.pdf} +\caption{High-level comparison of Hyperledger Fabric and Bitcoin, from \cite{belchior2020survey}} +\label{fig:bcs} \end{figure} -\section{Blockchain4Students} -A very reliable way to learn blockchain is to build one. Blockchain4Students is a blockchain\footnote{https://github.com/hyperledger-labs/university-course/tree/master/support/Lab03\%20-\%20Blockchain4Students} written in the Python programming language specifically tailored to be easily understandable, and thus be a reliable basis for blockchain teaching. Let's dive deeper into the blockchain world by exploring Blockchain4Students. \section{Hands on Blockchain} -%explain our prototype, show code - -%set up blockchain with 3 nodes and setup UI - -%questions on: -% node configuration -% node discovery -% block -% -% transactions -% pow and longest chain -% attacks (submit invalid block or invalid tx) -% extend the blockchain +A very reliable way to learn Blockchain is to build one. Blockchain4Students is a blockchain\footnote{https://github.com/hyperledger-labs/university-course/tree/master/support/Lab03\%20-\%20Blockchain4Students} written in the Python programming language specifically tailored to be easily understandable and thus be a reliable basis for blockchain teaching. Let's dive deeper into the blockchain world by exploring Blockchain4Students, and then answering to the following exercises. \subsection{Theory \& Business} -%private vs public \subsubsection*{Exercise 1: What is the advantage of the Blockchain being a decentralized, Peer-to-Peer system instead of a centralized one?} @@ -201,7 +186,7 @@ \subsubsection*{Exercise 6: How do we know if a transaction was created by a cli %PoW -\subsubsection*{Exercise 7: Why cant permissionless blockchains use the classical Consensus algorithms and must instead rely on other forms of Consensus like Proof-Of-Work?} +\subsubsection*{Exercise 7: Why can't permissionless blockchains use the classical Consensus algorithms and must instead rely on other forms of Consensus like Proof-Of-Work?} \subsubsection*{Exercise 8: Why can't a node fake a result from Proof-of-Work?} @@ -238,22 +223,22 @@ \subsubsection*{Exercise 1: Consider the next code fragment. What type is the Bl \begin{minted}[mathescape, - linenos, - numbersep=5pt, - gobble=2, - frame=lines, - framesep=2mm] - {python} +linenos, +numbersep=5pt, +gobble=2, +frame=lines, +framesep=2mm] +{python} \@app.route('/register/node', methods=['POST']) def register_peer_node(): - node_address = request.get_json()["node_address"] +node_address = request.get_json()["node_address"] - response = { - 'message': 'Node added', - 'total_nodes': list(blockchain.peer_nodes), - } - blockchain.register_node(node_address) - return json.dumps(response), 200 +response = { +'message': 'Node added', +'total_nodes': list(blockchain.peer_nodes), +} +blockchain.register_node(node_address) +return json.dumps(response), 200 \end{minted} @@ -261,15 +246,15 @@ \subsubsection*{Exercise 1: Consider the next code fragment. What type is the Bl \subsubsection*{Exercise 2: We give rewards right after a block is mined. What is the problem with this approach?} -\subsubsection*{Exercise 3: Each Transaction include the ID from the node that created it. Why is this information necessary?} +\subsubsection*{Exercise 3: Each Transaction includes the ID from the node that created it. Why is this information necessary?} -\subsubsection*{Exercise 4: When a transaction request arrives to the node, it signs it with its own private key. What is the possible attack with this approach?} +\subsubsection*{Exercise 4: When a transaction request arrives at the node, it signs it with its own private key. What is the possible attack with this approach?} -\subsubsection*{Exercise 5: When a node first connects, it contacts a discovery node to find other peers in the blockchain. Although this method is possible, why is it not realistic to use on wider scale blockchains? } +\subsubsection*{Exercise 5: When a node first connects, it contacts a discovery node to find other peers in the Blockchain. Although this method is possible, why is it not realistic to use on broader scale blockchains? } \subsubsection*{Exercise 6: When the node obtains a valid chain bigger than its own, it discards its own chain for the bigger one. Why? } -\subsubsection*{Exercise 7: With our current implementation, when a node joins he is not aware of any state of its peers. Add the functionality of when a node joins the system, to ask and update its own blockchain.} +\subsubsection*{Exercise 7: With our current implementation, when a node joins, he is not aware of any state of its peers. Add the functionality of when a node joins the system to ask and update its own Blockchain.} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT ENDS HERE diff --git a/source/Lab03/Instructors Guide/assignment.tex b/source/Lab03/Instructors Guide/assignment.tex index 15ae527..9803e92 100644 --- a/source/Lab03/Instructors Guide/assignment.tex +++ b/source/Lab03/Instructors Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -19,8 +19,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -70,16 +70,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 3\\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & - \\ +\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & - \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Background: A Primer on Blockchain Technology} & \textbf{Due Date:} & - \\ +\multicolumn{2}{|l|}{Background: A Primer on Blockchain Technology} & \textbf{Due Date:} & - \\ \hline \end{tabularx}} \rhead{} @@ -94,119 +94,117 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 \section*{Instructors Guide} -This document provides a proposal of solution for Lab3. This laboratory is based on previous work elaborated by the contributors \cite{belchior2019_audits,belchior2020,belchior2019_thesis}. +This document provides a proposal for a solution for Lab3. This laboratory is based on previous work elaborated by the contributors \cite{belchior2019_audits,belchior2020,belchior2019_thesis}. \section{Hands on Blockchain} \subsection{Theory \& Business} -%private vs public -\subsubsection*{Exercise 1: What is the advantage of the Blockchain being a decentralized, Peer-to-Peer system instead of a centralized one?} +\subsubsection*{Exercise 1: What is the advantage of blockchain being a decentralized, Peer-to-Peer system instead of a centralized one?} -By being a decentralized, Peer-to-Peer system, the chain becomes more resilient to integrity attacks and malicious changes to the blocks and contents, since the chain is verified by all the various entities and miners in the system. If we had a centralized system, it could become a single point of failure and of attack, since changing that "main" chain could alter the whole system. +Being a decentralized, peer-to-peer system, the chain becomes more resilient to integrity attacks and malicious action. This is due to blocks and their contents being verified. If we had a centralized system, it could become a single point of failure and attack since changing that ``main" chain could alter the whole system. \subsubsection*{Exercise 2: Consider a private blockchain running three nodes. Given that it is managed from a single organization, how do we assure decentralization?} -Although one organization controls the blockchain, decentralization can be achieved if there are stakeholders with different incentives, i.e., nodes representing different departments of a company. However, the executive management may enforce decisions on all departments, which can compromise decentralization. To deal with this, it is advisable that there is at least one other party, that is impartial, desirably. +Although one organization controls the blockchain, decentralization can be achieved if there are stakeholders with different incentives, i.e., nodes representing various company departments. However, executive management may enforce decisions on all departments, which can compromise decentralization. To deal with this, it is advisable that at least one other party is impartial, desirably. \subsubsection*{Exercise 3: What stops a malicious entity from being able to alter blocks in a permissionless blockchain?} -For permissionless blockchains, in order for a block to be accepted by other blocks, it must first complete the Proof-Of-Work, which is solving an hashing puzzle which includes the hash of the previous block. If an attacker were to alter a block, it would not only have to solve the Proof-Of-Work for that block as well as recalculate all Proof-Of-Works from subsequent blocks, since the Hash of the altered block has changed. Although not impossible, the deeper you go in the chain the more probabilistic unlikely it becomes, depending on the computational power the attacker has. +For permissionless blockchains, for a block to be accepted by other blocks, the entity who proposes it must participate in the consensus. For example, in Proof-Of-Work, the participant needs to solve a hashing puzzle that includes the hash of the previous block. If an attacker were to alter a block, it would have to solve the Proof-Of-Work for that block and recalculate all Proof-Of-Works from subsequent blocks the hash of the altered block has changed. Although not impossible, the deeper you go in the chain, the more probabilistic unlikely it becomes, depending on the computational power the attacker has. \subsubsection*{Exercise 4: What about permissioned blockchains? Can a malicious entity change in these as well?} -Since permissioned blockchains require credentials for participation and are usually more private, the presence of malicious entities is much less of a problem. Nevertheless, if we were to consider a malicious entity existed in the system, it would need a majority of malicious entities in the system in order to control the chain, since the Consensus algorithms are based on digital signatures. +Since permissioned blockchains require credentials for participation and are usually more private, the presence of malicious entities is much less of a problem. Nevertheless, if we were to consider a malicious entity existing in the system, it would need a majority of malicious entities to control the chain. %Txs \subsubsection*{Exercise 5: How does a client insert a transaction in the system? } -A client first inserts the transaction into a transaction pool, which each node has. This transaction pool is different for every node, since each client inserts transactions at nodes of his preference and these transactions may not be sent to every node in the system. +Firstly, a client inserts the transaction into a transaction pool, which each node has. This transaction pool is different for every node since each client inserts transactions at nodes of his preference, and these transactions may not be sent to every node in the system. \subsubsection*{Exercise 6: How do we know if a transaction was created by a client and not by a malicious node?} -A: The transaction must be signed with the private key of the sender in order to prove the authenticity and non-repudiation of the transaction. Hence, a malicious node cannot create transactions of its own accord. +A: The transaction must be signed with the sender's private key to prove the authenticity and non-repudiation of the transaction. Hence, a malicious node cannot create transactions of its own accord. %PoW -\subsubsection*{Exercise 7: Why cant permissionless blockchains use the classical Consensus algorithms and must instead rely on other forms of Consensus like Proof-Of-Work?} +\subsubsection*{Exercise 7: Why can't permissionless blockchains use the classical Consensus algorithms and must instead rely on other forms of Consensus like Proof-Of-Work?} -A: One of the main advantages of the permissionless blockchain is its openness to the internet, meaning anyone can enter and leave the system at will and participate as they so desire. Classical Consensus algorithms requires that the number of participants N is know, which is unfeasible to obtain in an open Peer-to-Peer system like a Blockchain. Furthermore, since it is open to the internet, there can be a number of byzantine clients present in the chain, which would be impossible to predict. Hence, Byzantine Consensus algorithms would not work as well, since we do not know either the number of participants (N) or the maximum number of possible faults/byzantine entities (F). +A: One of the main advantages of the permissionless blockchain is its openness to the internet, meaning anyone can enter and leave the system at will and participate as they so desire. Classical Consensus algorithms require that the number of participants N is known, which is unfeasible to obtain in an open Peer-to-Peer system like a Blockchain. Furthermore, since it is open to the internet, several byzantine clients are present in the chain, which would be impossible to predict. Hence, Byzantine Consensus algorithms would not work since we do not know either the number of participants (N) or the maximum number of possible faults/byzantine entities (F). \subsubsection*{Exercise 8: Why can't a node fake a result from Proof-of-Work?} A: Proof-of-Work relies on hashing functions. These functions have the following properties: \begin{itemize} - \item Even Distribution, meaning that there is an equal probability of obtaining any hash value, so we cannot predict which value we can obtain from a certain input; - \item One-way, meaning that it is hard to find the input value from the function result; - \item Collision resistance, meaning that it is hard to find a pair where inputs x and y, with x != y that hash(x) == hash(y). +\item Even Distribution, meaning that there is an equal probability of obtaining any hash value, so we cannot predict which value we can obtain from a certain input; +\item One-way, meaning that it is hard to find the input value from the function result; +\item Collision resistance, meaning that it is hard to find a pair where inputs x and y, with x != y that hash (x) == hash (y). \end{itemize} \subsubsection*{Exercise 9: Why do we want to add difficulty to the Proof-of-Work?} -A: By adding the difficulty level, we can control how fast the blockchain grows and new blocks are added. This is key in order to avoid malicious users to overwrite and change older blocks and the chain as a whole. If we add a difficulty of 0 (meaning a normal hash), it would be trivial for an attacker to simply remove a transaction from a block and recalculate all the hashes. By adding a difficulty equal to, for example, calculating an hash for a block every 10 minutes, it disincentivizes attackers to attack the chain, since they had to recalculate all the block hashes after the altered one and in that time, new blocks would appear, meaning that the chances of his new chain getting bigger than the others is very small. +A: By increasing the difficulty level, one can control how fast the blockchain grows and how quickly new blocks are added. This is key to avoid malicious users to overwrite and change older blocks and the chain as a whole. If we add a difficulty of 0 (meaning a normal hash), it would be trivial for an attacker to simply remove a transaction from a block and recalculate all the hashes. By adding a difficulty equal to, for example, calculating a hash for a block every 10 minutes, it disincentivizes attackers to attack the chain. Since they have to recalculate all the block hashes after the altered one and at that time, new blocks would appear. This means that the chances of its new chain getting longer than the others are smaller. \subsubsection*{Exercise 10: How does the difficulty value work in Proof-of-Work?} -The point of the difficulty value is to define how long it should take for a block to be created, given that with the advancement of software and hardware the computation power of a machine will be getting higher and the time to calculate/find the correct nonce value decreases. -Given that for hash functions, there is an Even Distribution, meaning that there is an equal probability of obtaining any hash value, the chances of calculating a specific hash (with X number of 0 in the beginning) is much lower, hence we can control the speed at which the blockchain evolves. +This question might mean different things: what is the purpose of the difficulty in proof-of-work, or how is it updated. The goal of the difficulty value is to define how long it should take for a block to be created. The higher the difficulty, the longer it takes to create a block. In Bitcoin, proof of work is configured to yield a block every 10 minutes. Hence, the difficulty is variable, depending on the number of miners on the network. %Block creation \subsubsection*{Exercise 11: How does a node share its new block?} -When a block is created, it can be shared with the other peers in many different ways. A node can either broadcast its chain once it finishes a new block in order to inform the other nodes or a node can periodically check the other nodes chains in order to know if new blocks/longer chains exist. +When a block is created, it can be shared with the other peers in many different ways. A node can either broadcast its chain once it finishes a new block to inform the other nodes. A node can periodically check the other nodes' chains to know if new blocks/longer chains exist. \subsubsection*{Exercise 12: What are the advantages and disadvantages of sharing the chain by broadcasting the block or by verifying other chains from peer nodes?} -Regarding the broadcasting method: +Regarding the broadcasting method: +\begin{itemize} +\item advantage \begin{itemize} - \item Advantage - \begin{itemize} - \item The node only needs to verify the block itself, since the chain behind it is already verified locally; - \end{itemize} - \item Disadvantage - \begin{itemize} - \item A node may not have the previous block from the one obtained, due to delays in the sharing of blocks; - \item All correctly verified block must be kept for the lifetime of the chain (or at least to a certain predicted depth), since we never know when a chain will become the new longest one; - \end{itemize} +\item The node only needs to verify the block itself, since the chain behind it is already verified locally; +\end{itemize} +\item Disadvantage +\begin{itemize} +\item A node may not have the previous block from the one obtained, due to delays in the sharing of blocks; +\item All correctly verified block must be kept for the lifetime of the chain (or at least to a certain predicted depth) since we never know when a chain will become the new longest one; +\end{itemize} \end{itemize} Regarding the verification of chains method: \begin{itemize} - \item Advantage - \begin{itemize} - \item Avoid problems of broadcasting, only relies on checking each neighboring known peer; - \end{itemize} - \item Disadvantage - \begin{itemize} - \item The node needs to verify the entirety of the chain, which includes verifying each signature of each block. Given that these signatures are with asymmetrical encryption, it will take a long time to verify the whole chain, especially the longest it becomes. - \end{itemize} +\item advantage +\begin{itemize} +\item Avoid problems of broadcasting, only relies on checking each neighboring known peer; +\end{itemize} +\item Disadvantage +\begin{itemize} +\item The node needs to verify each chain's entirety, which includes verifying each signature of each block. Given that these signatures are with asymmetrical encryption, it will take a long time to verify the whole chain, especially the longest it becomes. +\end{itemize} \end{itemize} \subsubsection*{Exercise 13: What happens when multiple nodes mine a new block at the same time?} -When two nodes create a new block, a temporary fork happens in the system. Both blocks and blockchains are legitimate, so nodes can choose to mine in either one of them, however the chain where a new block is mined first will be the one chosen by the miners for future work, hence the fork will resolve by itself with the mining of new blocks. +When two nodes create a new block, a temporary fork happens in the system. Both blocks and blockchains are legitimate, so nodes can choose to mine in either one of them. However the chain where a new block is mined first will be chosen by the miners for future work. Hence the fork will resolve by itself with the mining of new blocks. \subsubsection*{Exercise 14: Why do the miners choose the longest chain to work and not a forked, smaller chain?} -There are multiple reasons for this. From the blockchain perspective, the longer the chain is, the harder it is to alter past blocks, upholding the immutability of the chain. From the miner perspective, its goal is to obtain the reward for mining a block. If a miner is working on a block and a longer chain is discovered, the chances of his own chain being accepted by the other nodes gets smaller and the chances of a new block appearing on the longest chain gets higher. As such, working in the longest chain gives the highest chance of a block being accepted and upheld in the system, possibly giving the miner the reward. +There are multiple reasons for this. From the blockchain perspective, the longer the chain is, the harder it is to alter past blocks, upholding the chain's immutability. From the miner perspective, its goal is to obtain the reward for mining a block. Suppose a miner is working on a block, and a longer chain is discovered. In that case, the chances of his own chain being accepted by the other nodes get smaller, and the chances of a new block appearing on the longest chain get higher. Working in the longest chain gives the highest chance of a block being accepted and upheld in the system, possibly giving the miner the reward. \subsubsection*{Exercise 15: What happens to the other block and transactions that were created in the forked chain that was discontinued?} -Since the nodes and the system have started working on the longest chain and the chain discontinued, the block is considered invalid and the transactions are not accepted by the system. +Since the nodes and the system have started working on the longest chain and the chain discontinued, the block is considered invalid. The transactions are not accepted by the system. \subsubsection*{Exercise 16: What if the service was already made and the payment (in the form of a transaction) disappears?} -This is one of the challenges the blockchain must face. Since there is a chance the block may disappear from the chain due to forked, longer chain appearing, a transaction must only be considered “final” once the block reaches a certain depth. This depth must be chosen depending on the predicted computational power from the biggest entities on the system and the probability we want to uphold. +This is one of the challenges the blockchain must face. Since there is a chance the block may disappear from the chain due to forked, longer chain appearing, a transaction must only be considered “final” once the block reaches a certain depth. This depth must be chosen depending on the predicted computational power from the system's biggest entities and the probability we want to uphold. -\subsubsection*{Exercise 17: Being a Peer-to-Peer service, how does a node discover which nodes to connect to in order to join the system?} +\subsubsection*{Exercise 17: Being a Peer-to-Peer service, how does a node discover which nodes to connect to join the system?} -Depending on the Blockchain there are many ways these can be done. It can be by either joining a known address (which is not very good for a P2P system), by making a DNS request to known names or by accessing a known database where nodes insert their addresses. +Depending on the Blockchain, there are many ways these can be done. It can be by either joining a known address (which is not very good for a P2P system), making a DNS request to known names, or accessing a known database where nodes insert their addresses. \begin{figure}[H] - \centering - \includegraphics[scale=0.4]{figures/Probability.jpg} - \caption{Relation between computation power, probability of overtaking the blockchain and block depth} - \label{fig:bcs} +\centering +\includegraphics[scale=0.4]{figures/Probability.jpg} +\caption{Relation between computation power, probability of overtaking the blockchain and block depth} +\label{fig:bcs} \end{figure} @@ -216,28 +214,28 @@ \subsection{Blockchain4Students} \subsubsection*{Exercise 1: Consider the next code fragment. What type is the Blockchain4Students blockchain? } -Public, permissionless blockchain, because everyone with the code (publicly available) and internet connection can interact with it. However, the node discovery service can be considered centralized. +Public, permissionless blockchain, because everyone with the code (publicly available) and an internet connection can interact with it. However, the node discovery service can be considered centralized. \begin{minted}[mathescape, - linenos, - numbersep=5pt, - gobble=2, - frame=lines, - framesep=2mm] - {python} +linenos, +numbersep=5pt, +gobble=2, +frame=lines, +framesep=2mm] +{python} \@app.route('/register/node', methods=['POST']) def register_peer_node(): - node_address = request.get_json()["node_address"] +node_address = request.get_json()["node_address"] - response = { - 'message': 'Node added', - 'total_nodes': list(blockchain.peer_nodes), - } - blockchain.register_node(node_address) - return json.dumps(response), 200 +response = { +'message': 'Node added', +'total_nodes': list(blockchain.peer_nodes), +} +blockchain.register_node(node_address) +return json.dumps(response), 200 \end{minted} @@ -246,25 +244,25 @@ \subsubsection*{Exercise 2: We give rewards right after a block is mined. What i As the blockchain has probabilistic finality, it should wait for a certain number of blocks to be confirmed (e.g., In Bitcoin is 6, which corresponds to about an hour). -\subsubsection*{Exercise 3: Each Transaction include the ID from the node that created it. Why is this information necessary?} +\subsubsection*{Exercise 3: Each Transaction includes the ID from the node that created it. Why is this information necessary?} -Transactions must be digitally signed by the nodes in order to guarantee its authenticity and integrity. Without the node ID, the nodes would not be able to know which node signed what and what public key to use to verify the authenticity of the transaction. +Transactions must be digitally signed by the nodes to guarantee its authenticity and integrity. Without the node ID, the nodes would not know which node signed what and what public key to use to verify the authenticity of the transaction. -\subsubsection*{Exercise 4: When a transaction request arrives to the node, it signs it with its own private key. What is the possible attack with this approach?} +\subsubsection*{Exercise 4: When a transaction request arrives at the node, it signs it with its own private key. What is the possible attack with this approach?} -A malicious node could create fake transactions in order to benefit himself, like moving money from one wallet to another without the users consent or even to the miner itself. To solve this problems, when users request a new transaction, they should send a signature of their request to guarantee the authenticity of the transaction. +A malicious node could create fake transactions to benefit himself, like moving money from one wallet to another without the users' consent or even to the miner itself. To solve these problems, when users request a new transaction, they should send a signature of their request to guarantee the transaction's authenticity. -\subsubsection*{Exercise 5: When a node first connects, it contacts a discovery node to find other peers in the blockchain. Although this method is possible, why is it not realistic to use on wider scale blockchains? } +\subsubsection*{Exercise 5: When a node first connects, it contacts a discovery node to find other peers in the blockchain. Although this method is possible, why is it not realistic to use on broader scale blockchains? } -Blockchains are by definition a decentralized entity. By making a single node responsible for node discovery, we are not only losing this decentralization as we are adding a single point of failure to the blockchain. If this node fails, no other nodes would be able to connect. The more realistic way to deal with this problem is to add some form of name discovery, either by a DNS resolution (where failures could be masquerade and could point to multiple nodes) or other more advanced systems of name discovery like UDDI \footnote{https://en.wikipedia.org/wiki/Web\_Services\_Discovery} +Blockchains are, by definition, a decentralized entity. By making a single node responsible for node discovery, we are not only losing this decentralization as we are adding a single point of failure to the blockchain. If this node fails, no other nodes would be able to connect. The more realistic way to deal with this problem is to add some form of name discovery, either by a DNS resolution (where failures could masquerade and could point to multiple nodes) or other more advanced systems of name discovery like UDDI \footnote{https://en.wikipedia.org/wiki/Web\_Services\_Discovery} \subsubsection*{Exercise 6: When the node obtains a valid chain bigger than its own, it discards its own chain for the bigger one. Why? } -The goal of the miner (and the node) is to be the first to mine a block in order to obtain the reward from the system. If a miner is working on a block and it receives a chain with an already blocked mine further ahead, the chances of him finishing its current block and being accepted by the chain are very low. As such, a miner is better off giving up on their current work and starting a new block from the end of that chain. +The miner (and the node) is to be the first to mine a block to obtain the reward from the system. If a miner is working on a block and it receives a chain with an already blocked mine further ahead, the chances of him finishing its current block and being accepted by the chain are very low. As such, a miner is better off giving up on their current work and starting a new block from the end of that chain. -\subsubsection*{Exercise 7: With our current implementation, when a node joins he is not aware of any state of its peers. Add the functionality of when a node joins the system, to ask and update its own blockchain.} +\subsubsection*{Exercise 7: With our current implementation, when a node joins, he is not aware of any state of its peers. Add the functionality of when a node joins the system to ask and update its own blockchain.} -When a node joins, after node discovery, ask each node for the /getChain command, parse and verify the chain and if any is bigger substitute. +After node discovery, when a node joins, ask each node for the /getChain command, parse and verify the chain and if any is bigger substitute. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% diff --git a/source/Lab04/Guide/assignment.tex b/source/Lab04/Guide/assignment.tex index 5b364b4..9f34d84 100644 --- a/source/Lab04/Guide/assignment.tex +++ b/source/Lab04/Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -26,8 +26,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -77,16 +77,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 4 \\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & - \\ +\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & - \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Background: Introduction to Hyperledger Fabric} & \textbf{Due Date:} & - \\ +\multicolumn{2}{|l|}{Background: Introduction to Hyperledger Fabric} & \textbf{Due Date:} & - \\ \hline \end{tabularx}} \rhead{}} @@ -102,8 +102,8 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 \section*{Preliminary Notes} -We now introduce the Hyperledger Fabric blockchain, an enterprise grade permissioned distributed ledger platform that offers modularity and versatility for a broad set of industry use cases. -This laboratory is based on several sources \cite{belchior2019_thesis, fabric}. +We now introduce the Hyperledger Fabric blockchain, an enterprise-grade permissioned distributed ledger platform that offers modularity and versatility for a broad set of industry use cases. +This laboratory is based on several sources \cite{belchior2019_thesis, fabric}. %The reference section should be viewed as a ``additional readings reference'' - if you would like more information about the topic. @@ -111,106 +111,106 @@ \section*{Preliminary Notes} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT STARTS HERE -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{The Hyperledger Ecosystem} -Hyperledger is an open source community. Hyperledger stewards the development of frameworks, tools and libraries for enterprise-grade blockchain deployments. -It hosts Hyperledger Fabric, Sawtooth, Indy, as well as tools like Hyperledger Caliper, and Hyperledger Explorer. There are also tools hpsted at Hyperledger Labs \footnote{https://www.hyperledger.org/blog/2018/01/23/introducing-hyperledger-labs}, an open collaboration space, made by blockchain enthusiasts, that can benefit the Hyperledger ecosystem. An example is Hyperledger Umbra\footnote{https://github.com/hyperledger-labs/umbra}, allowing to run Hyperdger blockchains under Mininet for the purposes of testing scalability and consensus mechanisms. +\section{ Hyperledger Ecosystem} +Hyperledger is an open-source community. Hyperledger stewards the development of frameworks, tools, and libraries for enterprise-grade blockchain deployments. +It hosts Hyperledger Fabric, Sawtooth, Indy, and tools like Hyperledger Caliper and Hyperledger Explorer. There are also tools hosted at Hyperledger Labs \footnote{https://www.hyperledger.org/blog/2018/01/23/introducing-hyperledger-labs}, an open collaboration space made by blockchain enthusiasts, that can benefit the Hyperledger ecosystem. An example is Hyperledger Umbra\footnote{https://github.com/hyperledger-labs/umbra}, allowing to run Hyperdger blockchains under Mininet for the purposes of testing scalability and consensus mechanisms. \begin{figure}[h] - \includegraphics[scale=0.09]{figures/hl.png} - \centering - \caption{Hyperledger Greenhouse} - \label{fig:hl} +\includegraphics[scale=0.09]{figures/hl.png} +\centering +\caption{Hyperledger Greenhouse} +\label{fig:hl} \end{figure} -``Hyperledger launched in 2016 with a technical and organizational governance structure and 30 founding corporate members. Initially, the Hyperledger Technical Steering Committee welcomed two business blockchain framework codebases into incubation: Hyperledger Fabric, a codebase combining work by Digital Asset, libconsensus from Blockstream and OpenBlockchain from IBM; and Hyperledger Sawtooth, developed at Intel’s incubation group. Of the 70+ open source organizations the Linux Foundation has launched, Hyperledger holds the distinction as the fastest growing''\footnote{https://www.hyperledger.org/about}. +``Hyperledger launched in 2016 with a technical and organizational governance structure and 30 founding corporate members. Initially, the Hyperledger Technical Steering Committee welcomed two business blockchain framework codebases into incubation: Hyperledger Fabric, a codebase combining work by Digital Asset, libconsensus from Blockstream, and OpenBlockchain from IBM; and Hyperledger Sawtooth, developed at Intel’s incubation group. Of the 70+ open source organizations the Linux Foundation has launched, Hyperledger holds the distinction as the fastest-growing''\footnote{https://www.hyperledger.org/about}. -\section{The Hyperledger Fabric Blockchain} -\emph{Hyperledger Fabric (Fabric)}\footnote{https://www.hyperledger.org/use/fabric,https://hyperledger-fabric.readthedocs.io/en/release-2.2/} \cite{fabric} is a private blockchain framework, backed by the Linux Foundation and IBM, which is the main study object of this course. Hyperledger Fabric is a successful open-source project founded in 2016, counting with more than 13k commits, more than 250 contributors, and 10k stars in Github.\footnote{https://github.com/hyperledger/fabric} +\section{ Hyperledger Fabric Blockchain} +\emph{Hyperledger Fabric (Fabric)}\footnote{https://www.hyperledger.org/use/fabric,https://hyperledger-fabric.readthedocs.io/en/release-2.2/} \cite{fabric} is a private blockchain framework, backed by the Linux Foundation and IBM, which is the main study object of this course. Hyperledger Fabric is a successful open-source project founded in 2016, counting with more than 13k commits, more than 250 contributors, and 10k stars in Github.\footnote{https://github.com/hyperledger/fabric} Learning all about Hyperledger Fabric can open multiple windows of career prospects. Some of the top jobs trending in the United States of America are that of a Hyperledger Developer, Blockchain Engineer, Blockchain Solutions Architect, Project Manager, and jobs associated with web development (blockchain clients). ``As a qualified Hyperledger developer with good work experience, your salary package can fall anywhere between \$122–179k. But if you’re just starting out, it might vary between \$80–110k'' \footnote{https://medium.com/the-capital/how-to-become-a-hyperledger-developer-in-2020-bb19151e2a8}. -Hyperledger Fabric is also greatly popular in the academia. A search on Google Scholar with the terms ``Hyperledger Fabric'' returned 6,440 results (as of August 2020). Only in 2020, 1,710 papers were written. +Hyperledger Fabric is also greatly popular in academia. A search on Google Scholar with the terms ``Hyperledger Fabric'' returned 6,440 results (August 2020). Only in 2020, 1,710 papers were written. -Hyperledger Fabric is then a great resource to initiate your venture into the blockchain world, and at the same time learning about cryptography and distributed systems. +Hyperledger Fabric is then a great resource to initiate your venture into the blockchain world and learn about cryptography and distributed systems. -\subsection{A Technical View on Fabric} +\subsection{A Technical Viewpoint on Fabric} Along this section, we introduce Fabric and its components, how they are related, and how they work as a whole. Please be mindful of the contents learned during Lab1, Lab2, and Lab3. \subsection{Consensus} -Organizations group \emph{nodes (or peers, or peer nodes)} who form independent trust domains, i.e., peers from the same organization trust each other. Several organizations can form a \textit{consortium} -- a group of non-trusting organizations that want to achieve a business goal. +Organizations group \emph{nodes (or peers, or peer nodes)} who form independent trust domains, i.e., peers from the same organization trust each other. Several organizations can create a \textit{consortium} -- a group of non-trusting organizations that want to achieve a business goal. -Fabric allows for different kinds of participants in the network, which facilitates the \emph{execute-order-validate} paradigm for distributed execution of chaincode. Endorsement peers (\emph{endorsers}) execute (\emph{endorse}) the smart contracts (\emph{chaincode}) and return blockchain clients the validation output of submitted transactions, which contain the endorsement peers' signatures. This process allows parallel execution and addresses non-deterministic code. +Fabric allows for different kinds of participants in the network, which facilitates the \emph{execute-order-validate} paradigm for distributed execution of chaincode. Endorsement peers (\emph{endorsers}) execute (\emph{endorse}) the smart contracts (\emph{chaincode}) and return blockchain clients the validation output of submitted transactions, which contain the endorsement peers' signatures. This process allows parallel execution and addresses non-deterministic code. \begin{figure}[h] - \includegraphics[scale=0.6]{figures/execute_order_validate.png} - \centering - \caption{Execute-Order-Validate paradigm \cite{execute_order_validate}} - \label{fig:hf} +\includegraphics[scale=0.6]{figures/execute_order_validate.png} +\centering +\caption{Execute-Order-Validate paradigm \cite{execute_order_validate}} +\label{fig:hf} \end{figure} To achieve consensus, Fabric uses a permissioned voting-based scheme, which achieves low-latency. The \emph{endorsement policy} defines the voting-based scheme to be used by peers and, consequently, the weight of each peer regarding the validity of a transaction. In essence, the endorsement policies gather approvals for the execution of each transaction. -Apart from the consensus, Fabric needs to order blocks of transactions that can be proposed by different peers. The consensus on the block order is pluggable, i.e., using Kafka's Zookeeper (in previous 1.x.x Fabric versions) or RAFT (as in 2.x.x), both crash fault tolerant consensus mechanisms. Plugability allows the developer to adapt the consensus to the requirements of the environment. The consensus algorithms currently implemented to achieve consensus deterministically. This way, forks as in the Bitcoin network are prevented. Fabric can utilize different consensus protocols that do not require a native cryptocurrency, which reduces attack vectors and performance issues due to expensive operations, for instance, the ones required by proof of work. -Hyperledger Fabric added a novel way to deal with privacy and confidentiality, through its \textit{channel} architecture and private data feature\footnote{https://hyperledger-fabric.readthedocs.io/en/release-1.4/private-data/private-data.html}. +Apart from the consensus, Fabric needs to order blocks of transactions proposed by different peers. The consensus on the block order is pluggable, i.e., using Kafka's Zookeeper (in previous 1.x.x Fabric versions) or RAFT (as in 2.x.x), both crash fault-tolerant consensus mechanisms. Plugability allows the developer to adapt the consensus to the requirements of the environment. The consensus algorithms are currently implemented to achieve consensus deterministically. This way, forks as in the Bitcoin network are prevented. Fabric can utilize different consensus protocols that do not require a native cryptocurrency, which reduces attack vectors and performance issues due to expensive operations, for instance, the ones required by proof of work. +Hyperledger Fabric added a novel way to deal with privacy and confidentiality through its \textit{channel} architecture and private data feature\footnote{https://hyperledger-fabric.readthedocs.io/en/release-1.4/private-data/private-data.html}. -In conclusion, the consensus in Fabric is not limited to the agreed-upon order of a batch of transactions between peers, but rather, it is also the result of the execute-order-validate process that takes place during a transaction's flow from its proposal to its commitment on each peer's ledger. +In conclusion, the consensus in Fabric is not limited to the agreed-upon order of a batch of transactions between peers. Rather, it is also the result of the execute-order-validate process during a transaction's flow from its proposal to its commitment to each peer's ledger. \subsection{Chaincode} -Blockchain developers can write chaincode using a mainstream programming language, such as Javascript, GoLang or Java. Chaincode allows to implement various use cases. For instance, although Fabric does not have a built-in cryptocurrency, it is possible to create an underlying token with chaincode, which can represent assets or rights to perform specific actions. Such assets can be exchanged between network participants, through transactions. +Blockchain developers can write chaincode using a mainstream programming language, such as Javascript, GoLang, or Java. Chaincode allows for implementing various use cases. For instance, although Fabric does not have a built-in cryptocurrency, it is possible to create an underlying token with chaincode, representing assets or rights to perform specific actions. Such assets can be exchanged between network participants through transactions. \begin{figure}[h] - \includegraphics[scale=0.6]{figures/hf.png} - \centering - \caption{Hyperledger Fabric} - \label{fig:hf} +\includegraphics[scale=0.6]{figures/hf.png} +\centering +\caption{Hyperledger Fabric} +\label{fig:hf} \end{figure} -Chaincode is a key element in a Fabric network, as it dictates the rules to be followed by member participants. It is run in Docker containers, and is, thereby, isolated from the shared ledger. +Chaincode is a critical element of a Fabric network, as it dictates the rules to be followed by member participants. It is run in Docker containers and is, thereby, isolated from the shared ledger. -There are two types of chaincode: \emph{application chaincode}, that executes the application logic and communicates with the peers using \textit{gRPC messages}, and \emph{system chaincode}, ran on the \textit{configuration channel}. System chaincode assures the correction of chaincode execution (for example, that the endorsement policies are respected). Chaincode can be deployed dynamically, and it is usually running concurrently on the network. It runs directly on the peers' processes. The configuration channel stores the definition of MSPs (membership service providers), configuration about the consensus, ordering service parameters, and rules on how the channel configuration is tweakable. Chaincode executes transaction proposals against world state data. World state is a concept that represents the latest value for a key stored by a blockchain. By providing direct access to the latest value of these keys, there is no need to traverse the entire transaction log and calculate its values. Each peer contains a ledger component, formed by the \textit{block store}, which stores blocks containing transactions and the \textit{peer transaction manager (PTM)}. There is a different ledger for each channel, as channels enforce chaincode and data isolation. +There are two types of chaincode: \emph{application chaincode}, that executes the application logic and communicates with the peers using \textit{gRPC messages}, and \emph{system chaincode}, ran on the \textit{configuration channel}. System chaincode assures chaincode execution correction (for example, that the endorsement policies are respected). Chaincode can be deployed dynamically, and it is usually running concurrently on the network. It runs directly on the peers' processes. The configuration channel stores the definition of MSPs (membership service providers), configuration about the consensus, ordering service parameters, and rules on how the channel configuration is tweakable. Chaincode executes transaction proposals against world state data. The world state is a concept that represents the latest value for a key stored by a blockchain. By providing direct access to these keys' latest value, there is no need to traverse the entire transaction log and calculate its values. Each peer contains a ledger component, formed by the \textit{block store}, which stores blocks containing transactions and the \textit{peer transaction manager (PTM)}. There is a different ledger for each channel, as channels enforce chaincode and data isolation. \subsection{Peer nodes} Fabric defines on its model several kinds of peer nodes: \begin{itemize} - \item \emph{Committing peer (or simply peer)}. Each peer maintains the current snapshot of the current state of the ledger, as a key-value store. Such peers cannot invoke chaincode functions. - \item \emph{Endorser peer}. Endorser peers have chaincode installed. When they receive a transaction proposal, they simulate the transaction execution on isolated containers. Based on that simulation, such peers prepare a transaction proposal that is then sent to the orderer peer. The existence of endorser peers avoids sequential execution of transactions by all peers. - \item \emph{Orderer peer}. Orderers receive endorsed transactions and assemble them into blocks. After grouping transactions, orderers assure consensus, by propagating such blocks to committing peers, where they are validated and then committed to the shared ledger. Orderer peers record valid and invalid transactions, while other peers only contain valid transactions. - +\item \emph{Committing peer (or simply peer)}. Each peer maintains the current snapshot of the current state of the ledger as a key-value store. Such peers cannot invoke chaincode functions. +\item \emph{Endorser peer}. Endorser peers have chaincode installed. When they receive a transaction proposal, they simulate the transaction execution on isolated containers. Based on that simulation, such peers prepare a transaction proposal sent to the orderer peer. The existence of endorser peers avoids sequential execution of transactions by all peers. +\item \emph{Orderer peer}. Orderers receive endorsed transactions and assemble them into blocks. After grouping transactions, orderers assure consensus by propagating such blocks to committing peers. They are validated and then committed to the shared ledger. Orderer peers record valid and invalid transactions, while other peers only contain valid transactions. + \end{itemize} \begin{figure}[h] - \includegraphics[scale=1]{figures/peer_nodes.png} - \centering - \caption{Example of each node role \cite{peer_nodes}} - \label{fig:hf} +\includegraphics[scale=1]{figures/peer_nodes.png} +\centering +\caption{Example of each node role \cite{peer_nodes}} +\label{fig:hf} \end{figure} -Additionally, Fabric defines \emph{anchor peers} and \emph{leader peers}. Anchor peers serve as an intermediary between peers from its organisation and peers from an external one. Leader peers take the responsibility of distributing the transactions from the orderer to committing peers. +Additionally, Fabric defines \emph{anchor peers} and \emph{leader peers}. Anchor peers serve as an intermediary between peers from its organization and peers from an external one. Leader peers take the responsibility of distributing the transactions from the orderer to committing peers. \subsection{Channels} -Channels allow participants to establish a communication path between the subset of participants that have permissions to visualize a subset of transactions. For instance, in the same network, there can be a subset of peers that have access to only a certain kind of transactions. In addition to channels, Fabric supports \textit{private data}, which allows a defined subset of organizations on a channel to isolate their data from others. For example, when several traders engage in a blockchain ecosystem, it might be desirable for a seller to hide the price of their items from buyers\footnote{https://hyperledger-fabric.readthedocs.io/en/release-2.2/private-data/private-data.html}. In specific, organizations with permissions can endorse, commit, or query private data, which is logically separated from channel ledger data. In case of a dispute, private data can be shared and shown. For further privacy, hashes of private data go through the orderer, instead of the data itself. This is, the orderer does not access the transactional data to orderer it into blocks - but only its hash. It is disseminated peer-to-peer rather than via blocks. When transaction data must be kept confidential from ordering service nodes, it is a solution to use private data collections, rather than channels. +Channels allow participants to establish a communication path between the subset of participants who can visualize a subset of transactions. For instance, there can be a subset of peers in the same network that have access to only a certain kind of transaction. In addition to channels, Fabric supports \textit{private data}, which allows a defined subset of organizations on a channel to isolate their data from others. For example, when several traders engage in a blockchain ecosystem, it might be desirable for a seller to hide the price of their items from buyers\footnote{https://hyperledger-fabric.readthedocs.io/en/release-2.2/private-data/private-data.html}. In specific, organizations with permissions can endorse, commit, or query private data, which is logically separated from channel ledger data. In case of a dispute, private data can be shared and shown. For further privacy, hashes of private data go through the orderer instead of the data itself. The orderer does not access the transactional data to orderer it into blocks - but only its hash. It is disseminated peer-to-peer rather than via blocks. When transaction data must be kept confidential from ordering service nodes, it can use private data collections rather than channels. \begin{figure}[H] - \includegraphics[scale=0.3]{figures/channel.png} - \centering - \caption{A simple Fabric network} - \label{fig:channel} +\includegraphics[scale=0.3]{figures/channel.png} +\centering +\caption{A simple Fabric network} +\label{fig:channel} \end{figure} @@ -222,158 +222,158 @@ \subsection{Transaction Flow} The transaction flow, which follows the execute-order-validate paradigm, is depicted in Figure \ref{fig: Fabric's Transaction Flow}, is as it follows: \begin{figure}[] - \includegraphics[scale=0.85]{figures/back_hf_transaction_flow.png} - \centering +\includegraphics[scale=0.85]{figures/back_hf_transaction_flow.png} +\centering \caption{Transaction Flow \cite{fabric}} - \label{fig: Fabric's Transaction Flow} +\label{fig: Fabric's Transaction Flow} \end{figure} \begin{figure}[h] - \includegraphics[scale=0.5]{figures/transaction_flow.png} - \centering - \caption{Transaction Flow \cite{transaction_flow}} - \label{fig: Fabric's Transaction Flow} +\includegraphics[scale=0.5]{figures/transaction_flow.png} +\centering +\caption{Transaction Flow \cite{transaction_flow}} +\label{fig: Fabric's Transaction Flow} \end{figure} \begin{itemize} - \item \emph{Transaction proposal}. A blockchain client, which represents an organisation, creates a transaction proposal, and sends it to endorsement peers, as defined in the endorsement policy. The proposal contains information regarding the identity of the proposer, the transaction payload, a nonce, and a transaction identifier. - \item \emph{{Execute} (endorsement)}: the endorsement consists in the simulation of the transaction. The endorsers produce a write-set, containing the keys and their modified values, and a read-set. The endorsement peers execute the transactions, in an isolated environment. The endorsement is sent as the proposal response and contains the write-set, read-set, the transaction ID, endorser's ID, and the endorser's signature. When the client collects enough endorsements (which need to have the same execution result), it creates the transaction and sends it to the ordering service. The endorsement phase eliminates any eventual non-determinism. - -\item \emph{{Order}}: after the endorsement, there is the ordering phase, performed by orderers. The ordering service checks if the blockchain client that submitted the transaction proposal has appropriate permissions (broadcast and receiving permissions), on a given channel. Ordering produces blocks containing endorsed transactions, in an ordered sequence, per channel. The ordering allows the network to achieve consensus. The orderer broadcasts transaction's outputs to all the peers. For a correct ordering, there are some properties that the system must comply to, defined as: - - \begin{mydef}{Hash chain integrity:} - For any two blocks B delivered with sequence number s, and B' delivered with s' at correct peers such that s = s', - it holds B = B'. - \end{mydef} - - \begin{mydef}{Hash chain integrity:} - If some correct peer delivers a block B +\item \emph{Transaction proposal}. A blockchain client, which represents an organization, creates a transaction proposal and sends it to endorsement peers, as defined in the endorsement policy. The proposal contains information regarding the proposer's identity, the transaction payload, a nonce, and a transaction identifier. +\item \emph{{Execute} (endorsement)}: the endorsement consists in the simulation of the transaction. The endorsers produce a write-set, containing the keys and their modified values, and a read-set. The endorsement peers execute the transactions in an isolated environment. The endorsement is sent as the proposal response. It contains the write-set, read-set, the transaction ID, endorser's ID, and the endorser's signature. When the client collects enough endorsements (which need to have the same execution result), it creates the transaction and sends it to the ordering service. The endorsement phase eliminates any eventual non-determinism. + +\item \emph{{Order}}: after the endorsement, there is the ordering phase, performed by orderers. The ordering service checks if the blockchain client that submitted the transaction proposal has appropriate permissions (broadcast and receiving permissions) on a given channel. Ordering produces blocks containing endorsed transactions, in an ordered sequence, per channel. The ordering allows the network to achieve consensus. The orderer broadcasts the transaction's outputs to all the peers. For correct ordering, there are some properties that the system must comply with, defined as: + +\begin{mydef}{Hash chain integrity:} +For any two blocks, B delivered with sequence number s, and B' delivered with s' at correct peers such that s = s', +it holds B = B'. +\end{mydef} + +\begin{mydef}{Hash chain integrity:} +If some correct peer delivers a block B with number s and another correct peer delivers block B' = ([tx 1 , . . . , tx k ], h' ) with number s+1, then it holds h' = H (B), where H (·) denotes the cryptographic hash function. - \end{mydef} - - \begin{mydef}{No skipping:} +\end{mydef} + +\begin{mydef}{No skipping:} if a correct peer p delivers a block with number s $>$ 0 then peer p has already delivered all blocks with number < s. - \end{mydef} - - \begin{mydef}{No creation:} +\end{mydef} + +\begin{mydef}{No creation:} When a correct peer delivers block B with number s, -then for every tx $\in$ B some client has already broadcast tx. - \end{mydef} - - The ordering step assures all the above properties for each channel. The ordering service broadcasts blocks to the peers that maintain the state of the ledgers, via the ordering service or gossip protocol. +then for every tx $\in$ B, some client has already broadcast tx. +\end{mydef} + +The ordering step assures all the above properties for each channel. The ordering service broadcasts blocks to the peers that maintain the ledgers' state via the ordering service or gossip protocol. - \begin{mydef}{Validity:} +\begin{mydef}{Validity:} If a correct client invokes \textit{broadcast(tx)}, then every correct peer eventually delivers a block B that includes \textit{tx}, with some sequence number. \end{mydef} - - There is the need to note that the ordering service aims to achieve consensus and therefore prevents the network from forks. Consensus in Fabric encompasses the whole transaction flow, from the transaction proposal to the committing. It happens at the transaction level, where not all nodes need to engage in the consensus mechanism. Channels ensure that messages are delivered in the same logical order to committing peers. The ordering service achieves consensus in a deterministic way. As the ordering is deterministic, and not probabilistic, as in Bitcoin, forks do not occur. Permissioned blockchains which rely on the BFT replication protocols to achieve consensus can only support \textit{f} faulty nodes out of \textit{3f+1} nodes. This assumption may not match the trust model idealised for a specific application. The endorsement policy establishes the trust model, which is decoupled from the consensus mechanism, allowing developers to reason about the trust model independently of the consensus algorithm. - \item \emph{{Validate}}. Firstly, each peer validates the received transactions by checking if a transaction follows the correspondent endorsement policy. After that, a read-write conflict check is run against all transactions in the block, sequentially. For each transaction, it compares the versions of the keys in the read-set with those currently on the ledger. In case they do not match, the peers discard the transaction. Finally, the ledger is updated, in which the ledger appends the created block to its head. The ledger appends the results of the validity checks, including the invalid transactions. - +There is the need to note that the ordering service aims to achieve consensus and prevents the network from forks. Consensus in Fabric encompasses the whole transaction flow, from the transaction proposal to the committing. It happens at the transaction level, where not all nodes need to engage in the consensus mechanism. Channels ensure that messages are delivered in the same logical order to committing peers. The ordering service achieves consensus in a deterministic way. As the ordering is deterministic and not probabilistic, as in Bitcoin, forks do not occur. Permissioned blockchains that rely on the BFT replication protocols to achieve consensus can only support \textit{f} faulty nodes out of \textit{3f+1} nodes. This assumption may not match the trust model idealized for a specific application. The endorsement policy establishes the trust model, decoupled from the consensus mechanism, allowing developers to reason about the trust model independently of the consensus algorithm. + +\item \emph{{Validate}}. Firstly, each peer validates the received transactions by checking if a transaction follows the corresponding endorsement policy. After that, a read-write conflict check is run against all transactions in the block sequentially. It compares the versions of the keys in the read-set with those currently on the ledger for each transaction. In case they do not match, the peers discard the transaction. Finally, the ledger is updated, in which the ledger appends the created block to its head. The ledger appends the results of the validity checks, including the invalid transactions. + \end{itemize} \begin{figure}[ht] - \includegraphics[scale=.5]{figures/back_fabric_example_network.png} - \centering - \caption{Example of a Fabric's network} - \label{fig: Fabric's example network} +\includegraphics[scale=.5]{figures/back_fabric_example_network.png} +\centering +\caption{Example of a Fabric's network} +\label{fig: Fabric's example network} \end{figure} %COmponents Fabric has the following architectural components that support the transaction lifecycle: \begin{itemize} - \item \emph{Membership Service Provider}: the goal of this service is to identify participants (authentication) in the network uniquely. Public key infrastructure (\textit{PKI}) is used to generate certificates which are tied to members and organisations. Each organisation issues identities to its members and every peer recognise the members of the organisation. The identification is made through the association of a given peer to a cryptographic entity (i.e., digital certificate). Different identity management protocols can manage the identity of participants, such as LDAP and OpenID. - \item \emph{Ordering Service}: it manages multiple channels and comprises orderers and their processes. The batching process occurs in this phase, in which validated transactions are received and grouped into blocks. For each channel, after the transaction validation by the orderer, it broadcasts state updates to the ledger, in an atomic way, following a consensus algorithm. The ordering service can reconfigure channels and restrict broadcasting of transactions if needed. - \item \emph{Peer Gossip}: the peer gossip is responsible for broadcasting the results of the ordering phase, as well as \textit{transfer state} for unsynched peers (recently joined, after a downtime, or a peer is slower at validating the blocks before committing). Gossip data dissemination helps to achieve consistency and data integrity across nodes. Since blocks are signed and numbered, after a peer receives them, it can reconstruct the blockchain and verify its integrity. Thus, the gossip protocol manages peer discovery and channel membership and disseminates ledger data across peers on a channel. +\item \emph{Membership Service Provider}: This service aims to identify participants (authentication) in the network uniquely. Public key infrastructure (\textit{PKI}) is used to generate certificates that are tied to members and organizations. Each organization issues identities to its members, and every peer recognizes the members of the organization. The identification is made by associating a given peer to a cryptographic entity (i.e., digital certificate). Different identity management protocols can manage the identity of participants, such as LDAP and OpenID. +\item \emph{Ordering Service}: it manages multiple channels and comprises orderers and their processes. The batching process occurs in this phase, in which validated transactions are received and grouped into blocks. After the transaction validation by the orderer, it broadcasts state updates to the ledger, in an atomic way, following a consensus algorithm for each channel. The ordering service can reconfigure channels and restrict the broadcasting of transactions if needed. +\item \emph{Peer Gossip}: the peer gossip is responsible for broadcasting the results of the ordering phase, as well as \textit{transfer state} for unsynched peers (recently joined, after a downtime, or a peer is slower at validating the blocks before committing). Gossip data dissemination helps to achieve consistency and data integrity across nodes. Since blocks are signed and numbered, it can reconstruct the blockchain and verify its integrity after a peer receives them. Thus, the gossip protocol manages peer discovery and channel membership and disseminates ledger data across peers on a channel. \end{itemize} \subsection{Replication Model} -Fabric introduces a hybrid replication model, combining active and passive replication (primary-backup-replication, ported to the untrusted environment). Concerning active replication or state machine replication, the ledger state only reflects the transactions after these are validated and the consensus is reached, concerning their ordering. Passive replication happens as endorsers send the result of the transaction processing to commit nodes. Fabric comprises three main elements around data: \emph{world state}, a versioned key-value store which corresponds to the distributed ledger; the \emph{transaction log}, stores the history of all transactions (PTM); and a NoSQL database, such as CouchDB, stores the world state. It is possible to restrict users' access to view and edit specific fields and only authorising the read-only permissions. CouchDB supports complex data queries, comparatively to LevelDB, against the whole blockchain data, making it a suitable solution when it comes to data analysis and auditing. \emph{LevelDB} is the other built-in option for storing the world state. It is a simple, fast key-value storage library that provides an ordered mapping from string keys to string values. +Fabric introduces a hybrid replication model, combining active and passive replication (primary-backup-replication, ported to the untrusted environment). Concerning active replication or state machine replication, the ledger state only reflects the transactions after they are validated. The consensus is reached concerning their ordering. Passive replication happens as endorsers send the result of the transaction processing to commit nodes. Fabric comprises three main elements around data: \emph{world state}, a versioned key-value store which corresponds to the distributed ledger; the \emph{transaction log}, stores the history of all transactions (PTM); and a NoSQL database, such as CouchDB, stores the world state. It is possible to restrict users' access to view and edit specific fields and only authorizing the read-only permissions. CouchDB supports complex data queries, comparatively to LevelDB, against the whole blockchain data, making it a suitable solution for data analysis and auditing. \emph{LevelDB} is the other built-in option for storing the world state. It is a simple, fast key-value storage library that provides an ordered mapping from string keys to string values. -\section{Blockchain4Students - Hyperledger Fabric Version} - Education is a sector that may benefit greatly from blockchain. Today, most blockchain-based applications for eduation are directed towards supporting academic degree management and evaluation for learning outcomes, explored in projects such as Qualichain\footnote{https://qualichain-project.eu/}. QualiChain is a project funded by the EC under the Horizon 2020 program. The project aims to propose a blockchain-based approach for disrupting the archiving, management, and verification of educational and employment qualifications. In particular, QualiChain will support the storage, sharing, and verification of academic and other qualifications along with several additional services, provided by the platform. +\section{Blockchain4Students QUC - Hyperledger Fabric Version} +Education is a sector that may benefit greatly from the blockchain. Today, most blockchain-based applications for education are directed towards supporting academic degree management and evaluation for learning outcomes, explored in projects such as Qualichain\footnote{https://qualichain-project.eu/}. QualiChain is a project funded by the EC under the Horizon 2020 program. The project aims to propose a blockchain-based approach for disrupting the archiving, management, and verification of educational and employment qualifications. In particular, QualiChain will support the storage, sharing, and verification of academic and other qualifications and several additional services provided by the platform. -Besides credential management, blockchain can help streamlining formative and qualitative evaluation processes, by tracking details of teaching and learning, in a transparent way. In particular, formal education produces a lot of data, coming from several parties: student records, in-class data (attendance, grades), course data, payments, diplomas, and so on. %Challenges? -%Aggregating this information in a private way constitutes a challenge, and furthermore the data belongs to the school and not to the student. +Besides credential management, blockchain can streamline formative and qualitative evaluation processes by tracking teaching and learning details in a transparent way. In particular, formal education produces a lot of data, coming from several parties: student records, in-class data (attendance, grades), course data, payments, diplomas, and so on. %Challenges? +%Aggregating this information in a private way constitutes a challenge, and furthermore, the data belongs to the school and not to the student. -We now introduce a use case for education, \emph{Transparent Curricular Unit Quality System}, that serves as the basis for this course. This system is designed to improve the teaching conditions at universities, by enforcing accountability and transparency relatively to student's feedback on courses and its teaching staff. +We now introduce a use case for education, \emph{Transparent Curricular Unit Quality System}, that serves as the basis for this course. This system is designed to improve universities' teaching conditions by enforcing accountability and transparency relatively to students' feedback on courses and its teaching staff. \subsection{Transparent Curricular Unit Quality System} -Técnico Lisboa is a Portuguese public school of engineering and technology, part of University of Lisbon. University of Lisbon promotes continuous improvement via questionnaires, and the Course Unit Quality System (QUC) \cite{quc}. +Técnico Lisboa is a Portuguese public school of engineering and technology, part of the University of Lisbon. Unrelativef Lisbon promotes continuous improvement via questionnaires and the Course Unit Quality System (QUC) \cite{quc}. \begin{figure} - \centering - \includegraphics[scale=0.8]{figures/quc_3_1.PNG} - \caption{Questionnare to Técnico's Lisboa PhD students (in Portuguese): number of participants} - \label{fig:quc_3_1} +\centering +\includegraphics[scale=0.8]{figures/quc_3_1.PNG} +\caption{Questionnare to Técnico's Lisboa PhD students (in Portuguese): number of participants} +\label{fig:quc_3_1} \end{figure} \begin{figure} - \centering - \includegraphics[scale=0.6]{figures/quc_3_2.PNG} - \caption{Questionnare to Técnico's Lisboa PhD students (in Portuguese): development of scientific skills, and work and well-being management} - \label{fig:quc_3_1} +\centering +\includegraphics[scale=0.6]{figures/quc_3_2.PNG} +\caption{Questionnare to Técnico's Lisboa PhD students (in Portuguese): development of scientific skills, and work and well-being management} +\label{fig:quc_3_1} \end{figure} -Regarding questionnaires, Figures \ref{fig:quc_3_1} depicts a questionnaire answered by 36 PhD students on their experience at the same university. After processing, the questionnaires yield a list of action items aiming to improve the current status of the processes: propose a process that evaluates the quality of the theses' advising. This could be done via the QUCs. +Regarding questionnaires, Figures \ref{fig:quc_3_1} depicts a questionnaire answered by 36 Ph.D. students on their experience at the same university. After processing, the questionnaires yield a list of action items aiming to improve the processes' current status: propose a process that evaluates the quality of the theses' advising. This could be done via the QUCs. -QUCs are aimed at following up the functioning of each course unit, by promoting responsible involvement of students and teachers in the teaching, learning and assessment process. Students classify courses (and professors), in a scale from 1 to 9. The overall classification for both courses and professors is yielded. If professors or courses have low classifications, an audit can occur. +QUCs aim to follow up each course unit's functioning by promoting responsible involvement of students and teachers in the teaching, learning, and assessment process. Students classify courses (and professors) on a scale from 1 to 9. The overall classification for both courses and professors is yielded. If professors or courses have low classifications, an audit can occur. \begin{figure} - \centering - \includegraphics[scale=0.6]{figures/audits.PNG} - \caption{Audit timeline to the Artificial Intelligence course in the masters (MEIC) and bachelor (LEIC). The yellow square indicates that the course had a bad performance on QUC scores. The red box indicates that the course was audited.} - \label{fig:quc_3_1} +\centering +\includegraphics[scale=0.6]{figures/audits.PNG} +\caption{Audit timeline to the Artificial Intelligence course in the masters (MEIC) and bachelor (LEIC). The yellow square indicates that the course had a bad performance on QUC scores. The red box indicates that the course was audited.} +\label{fig:quc_3_1} \end{figure} -In the end of each semester, and for each Curricular Unit under an audit process, there will be presented the analysis of the process with the main conclusions, the suggested correction and implemented methods. +At the end of each semester, and for each Curricular Unit under an audit process, the analysis of the process will be presented with the main conclusions, the suggested correction, and implemented methods. -A professor or course should improve if their QUC median results is lower than 5. A professor is considered excellent if the median of their assessment factors is 9. Figure \ref{fig:quc1} depicts some questions appearing on the QUC questionnaires. +A professor or course should improve if their QUC median results are lower than 5. A professor is considered excellent if the median of their assessment factors is 9. Figure \ref{fig:quc1} depicts some questions appearing on the QUC questionnaires. \begin{figure} - \centering - \includegraphics[scale=0.6]{figures/quc_1.PNG} - \caption{Example of a section in a QUC questionnaire (in Portuguese). There are questions regarding the importance of different study materials in a specific course.} - \label{fig:quc1} +\centering +\includegraphics[scale=0.6]{figures/quc_1.PNG} +\caption{Example of a section in a QUC questionnaire (in Portuguese). There are questions regarding the importance of different study materials in a specific course.} +\label{fig:quc1} \end{figure} -This system aggregates feedback from both students and professors on the functioning of course units, providing a holistic view on the performance of universities, professors, and students. Students and professors are required to fill QUCs at the end of each semester. When a course or professor have a low classification, derived from students feedback, the university can intervene (audits/corrective measures). Such a system allows to continuous improvement, leading to a better educational system. +This system aggregates feedback from both students and professors on the functioning of course units, providing a holistic view of universities, professors, and students' performance. Students and professors are required to fill QUCs at the end of each semester. When a course or professor has a low classification, derived from students' feedback, the university can intervene (audits/corrective measures). Such a system allows for continuous improvement, leading to a better educational system. %problems -However, some problems arise: first of all, not all universities employ such a system. As such, consequences of bad teaching are often left unresolved, leading to a loss of human potential. Secondly, universities can tamper/hide QUC results, as they are the centralized party processing them. Having a common, global, QUC system that is able to make universities comparable in terms of course unit quality needs to be decentralized. If students directly vote on such platform, maintained by a consortium of high education institutions (such as Qualichain), the resiliency to the data greatly increases. Finally, the criteria and scoring result may not be transparent, in order universities to manipulate the perception about their quality. Therefore, there is the need for a standardized system that is clear, transparent, and simple. +However, some problems arise: first of all, not all universities employ such a system. As such, bad teaching consequences are often left unresolved, leading to a loss of human potential. Secondly, universities can tamper/hide QUC results, as they are the centralized party processing them. Having a common, global, QUC system that can make universities comparable in terms of course unit quality needs to be decentralized. If students directly vote on such a platform, maintained by a consortium of high education institutions (such as Qualichain), the resiliency to the data dramatically increases. Finally, the criteria and scoring result may not be transparent for universities to manipulate the perception about their quality. Therefore, there is a need for a standardized system that is clear, transparent, and simple. -\section{Blockchain4Students QUC - Fabric Version} +\section{Blockchain4Students QUC Implementation} To address those needs, we leverage the B4S-QUC system, based on Hyperledger Fabric. This system contains three stakeholders: universities, professors, and students. \begin{itemize} - \item Students vote on the parameters of teaching quality and course organization, via their local students group or association. - \item Professors give feedback on the course they taught via the professors department. - \item Universities analyze the results and apply corrective measures to courses. +\item Students vote on the parameters of teaching quality and course organization via their local student's group or association. +\item Professors give feedback on the course they taught via the professors' department. +\item Universities analyze the results and apply corrective measures to courses. -\end{itemize} +\end{itemize} -Feedback is collected from different questions, called \emph{feedback items}. The result of a set of questions towards a specific course is called a \emph{feedback entry}. Several feedback entries on a specific course compose the \emph{course feedback}, which is later analyzed to produce the QUC score (1-9). The score is applicable to the course and to the teaching staff of that course. +Feedback is collected from different questions, called \emph{feedback items}. The result of a set of questions towards a specific course is called a \emph{feedback entry}. Several feedback entries on a specific course compose the \emph{course feedback}, which is later analyzed to produce the QUC score (1-9). The score applies to the course and to the teaching staff of that course. -Let us think about a possible implementation. B4S QUC should implement a Transparent Curricular Unit Quality System, in which students give their feedback on courses/professors of their university through a students group. Schools and universities are rewarded reputation tokens based on the quality of their students' feedback. This allows to make professors directly accountable for their students -- yielding incentives for the universities to provide support for their improvement. However, some issues must be dealt with. +Let us think about possible implementation. B4S QUC should implement a Transparent Curricular Unit Quality System. Students give their feedback on courses/professors of their university through a student group. Schools and universities are rewarded reputation tokens based on the quality of their students' feedback. This allows making professors directly accountable for their students, yielding incentives for the universities to support their improvement. However, some issues must be dealt with. -\subsection{Exercises: B4S QUC} +\subsection{Exercises: B4S QUC} -Consider the Fabric version of Blockchain4Students - a blockchain dedicated to improve continuous improvement at education. +Consider the Fabric version of Blockchain4Students - a blockchain dedicated to improving continuous improvement in education. \subsubsection{Threats to Privacy} @@ -391,29 +391,29 @@ \subsubsection*{Exercise 4: What would a suitable endorsement policy be, for the \subsubsection{Data Integrity Threats} \label{sec: threat_model} -In order to mitigate the data integrity threats, it is important to perform an analysis of current challenges and risks. -We provide a general threat model with regard to data integrity in information systems and data integrity threats on the blockchain. Threats related to the blockchain, are based on our previous work \cite{belchior2019_audits}. +To mitigate data integrity threats, it is important to perform an analysis of current challenges and risks. +We provide a general threat model concerning data integrity in information systems and data integrity threats on the blockchain. Threats related to the blockchain are based on our previous work \cite{belchior2019_audits}. %reescrever -This section presents the threats to which data is exposed (in this case the 'feedback entries). The threat model will focus on three factors: i) internal access attacks ii) remote attacks and iii) Hyperledger Fabric blockchain attacks. +This section presents the threats to which data is exposed (in this case, the 'feedback entries). The threat model will focus on three factors: i) internal access attacks ii) remote attacks, and iii) Hyperledger Fabric blockchain attacks. -We assume that there are two types of adversaries. A third-party adversary, \emph{Adversary A$_{t}$}, can access the trusted computing base and remotely penetrate the system, by exploiting security flaws or by hijacking root user credentials. Stemming from this, A$_{t}$ might tamper data to compromise auditing and forensic procedures or tamper system data to cause damage to the organization. On the other hand, an adversary from within the organization, \emph{Adversary A$_{o}$} can be an employee with root privileges. Such a person can access the databases and can be motivated to tamper data for personal gains. Such adversaries can tamper data to hide their traces or tamper data to aid third-parties illegally. +We assume that there are two types of adversaries. A third-party adversary, \emph{Adversary A$_{t}$}, can access the trusted computing base and remotely penetrate the system by exploiting security flaws or by hijacking root user credentials. Stemming from this, A$_{t}$ might tamper data to compromise auditing and forensic procedures or tamper system data to damage the organization. On the other hand, an adversary from within the organization, \emph{Adversary A$_{o}$} can be an employee with root privileges. Such a person can access the databases and can be motivated to tamper with data for personal gains. Such adversaries can tamper with data to hide their traces or tamper data to aid third-parties illegally. -There can be two main types of attacks to data integrity: the \emph{physical access attack} and the \emph{remote vulnerability attack}. In the physical access attack, the adversary \emph{Adversary A$_{t}$} or \emph{Adversary A$_{o}$} have access to the critical system components. The attacker generates transactions to change the current values of the objects held in a database. As objects are being updated, the database generates an audit log, tracking the changes made by the attacker. The attacker then focuses on deleting the evidence by deleting the generated audit logs or changing its content. The attacker can manipulate the auditing process by modifying the history maintained by the audit log. +There can be two main types of attacks to data integrity: the \emph{physical access attack} and the \emph{remote vulnerability attack}. In the physical access attack, the adversary \emph{Adversary A$_{t}$} or \emph{Adversary A$_{o}$} has access to the critical system components. The attacker generates transactions to change the current values of the objects held in a database. As objects are being updated, the database generates an audit log, tracking the attacker's changes. The attacker then focuses on deleting the evidence by deleting the generated audit logs or changing its content. The attacker can manipulate the auditing process by modifying the history maintained by the audit log. -As a consequence, the obfuscation of illegal activities and impersonating someone's actions can occur. In remote vulnerability attacks, the attacker exploits default vulnerabilities on systems, such as software malfunctions and security vulnerabilities (e.g., SQL-injection). Although such attacks are common, corporate organizations have their systems and databases secure against conventional attacks. The research is focused on the physical access attacks. Five threats compose the threat model \cite{belchior2019_audits}, which are distributed through the various exercises: +As a consequence, the obfuscation of illegal activities and impersonating someone's actions can occur. In remote vulnerability attacks, the attacker exploits default vulnerabilities on systems, such as software malfunctions and security vulnerabilities (e.g., SQL-injection). Although such attacks are common, corporate organizations have their systems and databases secure against conventional attacks. The research is focused on physical access attacks. Five threats compose the threat model \cite{belchior2019_audits}, which are distributed through the various exercises: \subsubsection*{Exercise 5: Elaborate on the following threats:} -\begin{threat}{Feedback item tampering from an external element:} An attacker violates the integrity of the 'feedback entries, by editing them. +\begin{threat}{Feedback item tampering from an external element:} An attacker violates the integrity of the 'feedback entries by editing them. \end{threat} -\begin{threat}{Feedback item tampering from an internal adversary:} An attacker from one of the stakeholders violates the integrity of the 'feedback entries, by editing them directly. +\begin{threat}{Feedback item tampering from an internal adversary:} An attacker from one of the stakeholders violates the integrity of the 'feedback entries by editing them directly. \end{threat} -\begin{threat}{Feedback item tampering by the system administrator:} The administrator of the system, with the highest permissions, violates the integrity of the 'feedback entries, by editing them. There is the possibility of obfuscating activity traces by deleting evidence on other systems +\begin{threat}{Feedback item tampering by the system administrator:} The administrator of the system, with the highest permissions, violates the integrity of the 'feedback entries by editing them. There is the possibility of obfuscating activity traces by deleting evidence on other systems \end{threat} diff --git a/source/Lab04/Instructors Guide/assignment.tex b/source/Lab04/Instructors Guide/assignment.tex index a4f768f..4f7d4b3 100644 --- a/source/Lab04/Instructors Guide/assignment.tex +++ b/source/Lab04/Instructors Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -19,8 +19,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -70,16 +70,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 4\\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & - \\ +\multicolumn{2}{|l|}{Module I - Introduction} & \textbf{Issue Date:} & - \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Background: Introduction to Hyperledger Fabric} & \textbf{Due Date:} & - \\ +\multicolumn{2}{|l|}{Background: Introduction to Hyperledger Fabric} & \textbf{Due Date:} & - \\ \hline \end{tabularx}} \rhead{} @@ -94,16 +94,16 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 \section*{Instructors Guide} -This document provides a proposal of solution for Lab4 \cite{belchior2019_audits}. +This document provides a proposal for a solution for Lab4 \cite{belchior2019_audits}. -\section{B4S QUC - Fabric Version} +\section{B4S QUC - Fabric Version} -Consider the Fabric version of Blockchain4Students - a blockchain dedicated to improve continuous improvement at education. +Consider the Fabric version of Blockchain4Students - a blockchain dedicated to improving continuous improvement in education. -Let us think about a possible implementation. B4S QUC should implement a Transparent Curricular Unit Quality System, in which students give their feedback on courses/professors of their university through a students group. Schools and universities are rewarded reputation tokens based on the quality of their students' feedback. This allows to make professors directly accountable for their students -- yielding incentives for the universities to provide support for their improvement. +Let us think about possible implementation. B4S QUC should implement a Transparent Curricular Unit Quality System. Students give their feedback on courses/professors of their university through a student group. Schools and universities are rewarded reputation tokens based on the quality of their students' feedback. This allows making professors directly accountable for their students, yielding incentives for the universities to support their improvement. @@ -115,51 +115,51 @@ \subsection{Threats to Privacy} \subsubsection*{Exercise 1: Can a centralized system implement the use case?} -A: No. A centralized system like Tecnico's may alter the results of the evaluations, or collude with participants, due to a wide range of reasons. In other words, the intermediary between the feedback (students, professors) and the public (student candidate, high school student, other higher education institutions) is the university. This poses threats, as the institution is biased towards showing a good image. Via a blockchain, students and professors directly report feedback to the network, via their representing parties, allowing for better transparency, accountability, and decentralization. +A: No. A centralized system like Tecnico's may alter the evaluations' results or collude with participants due to a wide range of reasons. The intermediary between the feedback (students, professors), and the public (student candidate, high school student, other higher education institutions) is the university. This poses threats, as the institution is biased towards showing a good image. Via a blockchain, students and professors directly report feedback to the network via their representing parties, allowing for better transparency, accountability, and decentralization. \subsubsection*{Exercise 2: Which privacy concerns arise regarding this network?} -Given a students feedback, there is the danger of students being identified by the professor or university. The feedback should be given in an anonymous and confidential way. +Given students feedback, there is the danger of students being identified by the professor or university. The feedback should be given anonymously and confidentially. \subsubsection*{Exercise 3: How can one assure privacy and confidentiality for students and professors, without sacrificing traceability?} -A student ID is registered on the students group, which is associated with each feedback. In case of dispute, the law can enforce the students group to identity the student. Otherwise, in normal conditions, the id is not disclosed and privacy is assured. +A student ID is registered on the students' group, which is associated with each feedback. In case of dispute, the law can enforce the students' group to identify the student. Otherwise, in normal conditions, the id is not disclosed, and privacy is assured. \subsubsection*{Exercise 4: What would a suitable endorsement policy be, for the studied ecosystem?} -A: at least an independent organization approving transactions; n out out of 2n signatures from universities; k out of 2k signatures from student groups; i out of 2i signatures from professors departments +A: at least an independent organization approving transactions; n out of 2n signatures from universities; k out of 2k signatures from student groups; i out of 2i signatures from professors departments \subsection{Data Integrity Threats} \subsubsection*{Exercise 5: Elaborate on the following threats:} -\begin{threat}{feedback entry tampering from an external element:} An attacker violates the integrity of the feedback entries, by editing them. +\begin{threat}{feedback entry tampering from an external element:} An attacker violates the integrity of the feedback entries by editing them. \end{threat} - -Threat 1 (T1) is an external adversary gaining access to the feedback entries. Attackers can edit the feedback entries at their will, i.e., deleting them or decreasing the score. This attack has a higher severity on information systems which do not replicate data, as the attacker can permanently delete information. -\begin{threat}{feedback entry tampering from an internal adversary:} An attacker from one of the stakeholders violates the integrity of the feedback entries, by editing them directly. +Threat 1 (T1) is an external adversary gaining access to the feedback entries. Attackers can edit the feedback entries at their will, i.e., deleting them or decreasing the score. This attack has a higher severity of information systems that do not replicate data. The attacker can permanently delete information. + +\begin{threat}{feedback entry tampering from an internal adversary:} An attacker from one of the stakeholders violates the integrity of the feedback entries by editing them directly. \end{threat} Threat 2 (T2) is similar to T1, with a higher degree of severity. If adversaries are insiders, they have direct access to the protected resource. An internal adversary might know peculiar ways to obfuscate such activities. -\begin{threat}{feedback entry tampering by the system administrator:} The administrator of the system, with the highest permissions, violates the integrity of the feedback entries, by editing them. There is the possibility of obfuscating activity traces by deleting evidence on other systems. +\begin{threat}{feedback entry tampering by the system administrator:} The administrator of the system, with the highest permissions, violates the integrity of the feedback entries by editing them. There is the possibility of obfuscating activity traces by deleting evidence on other systems. \end{threat} -Threat 3 (T3) is similar to T2, with higher severity. Administrators have access to all resources and can, theoretically, delete all traces. +Threat 3 (T3) is similar to T2, with higher severity. Administrators have access to all resources and can, theoretically, delete all traces. \begin{threat}{A participant edits feedback entries that are protected by the blockchain.} \end{threat} -T4 is not severe, if using a permissioned, private blockchain, like Hyperledger Fabric (Fabric), because transactions have to be endorsed before committed. Even if any participant on the network tries to modify feedback entries maliciously on its ledger, they cannot change other peers' ledger state, as honest endorsers would not endorse such transactions. +T4 is not severe if using a permissioned, private blockchain, like Hyperledger Fabric (Fabric), because transactions have to be endorsed before committed. Even if any participant on the network tries to modify feedback entries maliciously on its ledger, they cannot change other peers' ledger state, as honest endorsers would not endorse such transactions. \begin{threat}The majority of participants conspire and modify the feedback entries. \end{threat} -T5 evolves from T4, where the minority of nodes try to tamper feedback entries. Members from the network can collude to alter the integrity of feedback entries, to trick an external auditor. If all participants on a network want to change its state, it is theoretically possible. The participants can follow the protocol and rewrite the world state, submitting new transactions from the point on in which they want to change the state. Another way to mitigate this threat is to periodically sent the last hash block of the permissioned blockchain, as a transaction, to a public blockchain at the expense of paying fees. +T5 evolves from T4, where the minority of nodes try to tamper with feedback entries. Members from the network can collude to alter feedback entries' integrity to trick an external auditor. If all participants on a network want to change its state, it is theoretically possible. The participants can follow the protocol and rewrite the world state, submitting new transactions from the point they want to change the state. Another way to mitigate this threat is to periodically send the last hash block of the permissioned blockchain, as a transaction, to a public blockchain at the expense of paying fees. \subsubsection*{Exercise 6: How to alleviate the mentioned threats?} -A: Threats 1-3 are alleviated by using a blockchain. Threat 4 is alleviate by Fabric's endorsement policies. Threat 5 is solved by adding a notary organization that is independent of any university. +A: Threats 1-3 are alleviated by using a blockchain. Threat 4 is alleviated by Fabric's endorsement policies. Threat 5 is solved by adding a notary organization that is independent of any university. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \bibliographystyle{IEEEtran} diff --git a/source/Lab05/Guide/assignment.tex b/source/Lab05/Guide/assignment.tex index 84a5e77..a4104af 100644 --- a/source/Lab05/Guide/assignment.tex +++ b/source/Lab05/Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -28,8 +28,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -79,110 +79,110 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline + \hline % In the following line change Course Name: PPIII, PPB -\textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 5 \\ -\hline + \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 5 \\ + \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module II - Hyperledger Fabric} & \textbf{Issue Date:} & - \\ -\hline + \multicolumn{2}{|l|}{Module II - Hyperledger Fabric} & \textbf{Issue Date:} & - \\ + \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Hyperledger Fabric: Infrastructure and Chaincode} & \textbf{Due Date:} & - \\ -\hline + \multicolumn{2}{|l|}{Hyperledger Fabric: Infrastructure and Chaincode} & \textbf{Due Date:} & - \\ + \hline \end{tabularx}} \rhead{}} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % DO NOT CHANGE THIS BLOCK \begin{document} -\textblockorigin{-34pt}{-12pt} -\begin{textblock*}{10cm}(2cm,1cm) -\includegraphics[width=6cm]{hyperledger.png} -\end{textblock*} -\newtheorem{mydef}{Definition} + \textblockorigin{-34pt}{-12pt} + \begin{textblock*}{10cm}(2cm,1cm) + \includegraphics[width=6cm]{hyperledger.png} + \end{textblock*} + \newtheorem{mydef}{Definition} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 -\section*{Preliminary Notes} + \section*{Preliminary Notes} -The majority of this laboratory was inspired on the implementation of JusticeChain\cite{belchior2019_audits}. + The majority of this laboratory was inspired by the implementation of JusticeChain\cite{belchior2019_audits}. -Working on the blockchain ecosystem includes two major components: smart contract (chaincode) development, and operations (infrastructure). We will learn how to create the chaincode to implement the system B4S QUC. + The blockchain ecosystem includes two major components: smart contract (chaincode) development and operations (infrastructure). We will learn how to create the chaincode to implement the system B4S QUC. -Contrarily to previous laboratories, you will take a look at the code and experiment with our system while you navigate through this guide. -To do so, please clone the repository at, using the command: -{git clone https://github.com/hyperledger-labs/university-course}, and navigate to support/Lab05 + Contrarily to previous laboratories, you will take a look at the code and experiment with our system while you navigate through this guide. + To do so, please clone the repository at, using the command: + {git clone https://github.com/hyperledger-labs/university-course}, and navigate to support/Lab05 %The reference section should be viewed as a ``additional readings reference'' - if you would like more information about the topic. -This lab uses carbon.sh to create some figures. + This lab uses carbon.sh to create some figures. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT STARTS HERE -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{The B4S QUC System} -< -Figure \ref{fig:actors} illustrates the different actors in this system. + \section{ B4S QUC System} + < + Figure \ref{fig:actors} illustrates the different actors in this system. + + \begin{figure}[h] + \centering + \includegraphics[scale=0.6]{figures/fig1.pdf} + \caption{Actors in a B4S QUC blockchain} + \label{fig:actors} + \end{figure} -\begin{figure}[h] - \centering - \includegraphics[scale=0.6]{figures/fig1.pdf} - \caption{Actors in a B4S QUC blockchain} - \label{fig:actors} -\end{figure} + However, since Fabric can only hold a limited number of peer nodes, and for privacy purposes, both students and professors use proxies in the communications. In other words, students interact with B4S QUC via the students union of their university, while professors + do the same via their department, as illustrated in Figure 2. -However, since Fabric can only hold a limited number of peer nodes, and for privacy purposes, both students and professors use proxies in the communications. In other words, students interact with B4S QUC via their students union of their university, while professors -do the same via their department, as illustrated in Figure 2. + \begin{figure}[h] + \centering + \includegraphics[scale=0.6]{figures/fig2.pdf} + \caption{Blockchain and Blockchain Client components in a B4S QUC blockchain} + \label{fig:actors_detailed} + \end{figure} -\begin{figure}[h] - \centering - \includegraphics[scale=0.6]{figures/fig2.pdf} - \caption{Blockchain and Blockchain Client components in a B4S QUC blockchain} - \label{fig:actors_detailed} -\end{figure} + The students union and department web applications are not hosted by the blockchain (we call those components off-chain components). Therefore, the students and professors are blockchain users that use the students' union peer node and the professors' department peer node. -The students union and departments web applications are not hosted by the blockchain (we call those components off-chain components). The students and professors are, therefore, blockchain users that use the students' union peer node, and the professors department peer node. + Next, we focus on each component of our system, namely the chaincode, the off-chain components (application, user interface), and the infrastructure. -Next, we focus on each component of our system, namely the chaincode, the off-chain components (application, user interface), and the infrastructure. + \subsection{States and transactions} + A state is composed of the stored attributes. Let’s analyze our use case in a little more detail. B4S QUC participants such as students and professors from Técnico Lisboa use transactions to achieve their business objectives. -\subsection{States and transactions} - A state is composed by the stored attributes. Let’s analyze our use case in a little more detail. B4S QUC participants such as students and professors from Técnico Lisboa use transactions to achieve their business objectives. - - A Fabric state is implemented as a key/value pair, in which the value encodes the object properties in a format that captures the objects' multiple properties, typically JSON. The ledger database can support advanced query operations against these properties, which is very helpful for sophisticated object retrieval. A key can be simple (formed by, for example, the id of a feedback item), or composed (the concatenation of the feedback item id + student id). A key therefore identifies unequivocally a state. We model a state like this in the following manner: - \begin{verbatim} + A Fabric state is implemented as a key/value pair. The value encodes the object properties in a format that captures the objects' multiple properties, typically JSON. The ledger database can support advanced query operations against these properties, which helps sophisticated object retrieval. A key can be simple (formed by, for example, the id of a feedback item) or composed (the concatenation of the feedback item id + student id). A key, therefore, identifies a state unequivocally. We model a state like this in the following manner: + \begin{verbatim} OBJECT KEY FIELD: VALUE FIELD: VALUE FIELD: VALUE ... FIELD: VALUE -\end{verbatim} - -Let’s examine the an example of a feedback item: + \end{verbatim} + + Let’s examine the an example of a feedback item: -\begin{verbatim} + \begin{verbatim} FeedbackItem 011a8b02-e9b7-4023-9be6-4814abf5664c -Creator: Student -Course ID: 197b067b-44fc-43b1-8f8c-1658de7ec185 0808f130-1ba8-4712-991a-250080e2e44c +Creator: Student +Course ID: 197b067b-44fc-43b1-8f8c-1658de7ec185 0808f130-1ba8-4712-991a-250080e2e44c FeedbackParameter#1: 5 FeedbackParameter#2: 9 FeedbackParameter#3: 2 Produced QUC: null Current State: Completed -\end{verbatim} + \end{verbatim} -The creator of a feedback item may be a student or a professor. Each feedback item is uniquely identified by a Universally unique identifier v4 (UUIDv4) string. Essentially, a UUID is a mechanism to identify information, composed by a 128-bit string. The probability to find a duplicate within 103 trillion version-4 UUIDs is one in a billion. FeedbackParameters are the QUC parameters, which we omit for simplicity (an example would be: Professor\_Arrives\_In\_Time). The produced QUC attribute refers to the QUC in which that feedback item participated. As the feedback hasn't been processed, no QUC was generated with that feedback item. + The creator of a feedback item may be a student or a professor. Each feedback item is uniquely identified by a Universally unique identifier v4 (UUIDv4) string. Essentially, a UUID is a mechanism to identify information composed of a 128-bit string. The probability of finding a duplicate within 103 trillion version-4 UUIDs is one in a billion. FeedbackParameters are the QUC parameters, which we omit for simplicity (an example would be: Professor\_Arrives\_In\_Time). The produced QUC attribute refers to the QUC in which that feedback item participated. As the feedback hasn't been processed, no QUC was generated with that feedback item. -We can differentiate on four types of feedback: feedback from students towards a course; feedback from students towards the teaching staff; feedback from professors toward the course; feedback from the students representative towards courses and teaching staff. However, for illustration and brevity, we focus on the feedback from a student towards a course. Several feedback items are used to produce a QUC score, which refers to a course and its teaching staff: + We can differentiate on four types of feedback: feedback from students towards a course, feedback from students towards the teaching staff; feedback from professors toward the course; feedback from the students representative towards courses, and teaching staff. However, for illustration and brevity, we focus on the feedback from a student towards a course. Several feedback items are used to produce a QUC score, which refers to a course and its teaching staff: -\begin{verbatim} + \begin{verbatim} QUC c38eea64-f494-49d0-8245-3966f02d8e4a University: University 0177545f-2730-496d-8971-6204ff804979 Teaching Staff: Professor d023fff3-1613-47db-a4ab-b91925ae1a36, Professor 90ea3765-1355-45a9-bc6e-3069520c372b @@ -198,119 +198,119 @@ \subsection{States and transactions} General Score: 6 Current State: Created -\end{verbatim} + \end{verbatim} -An example of a state for a course is: + An example of a state for a course is: -\begin{verbatim} + \begin{verbatim} Course b143ef55-64b7-41b5-b91b-0b8161f98ef6 University: University 0177545f-2730-496d-8971-6204ff804979 Teaching Staff: Professor d023fff3-1613-47db-a4ab-b91925ae1a36, Professor 90ea3765-1355-45a9-bc6e-3069520c372b Students: [...] Metadata: [...] Current State: Created -\end{verbatim} + \end{verbatim} -A course is taught by teaching staff, which later can also provide feedback from the course. The Students allows a student to prove that they were enrolled in that course - and thus being able to provide feedback. + A course is taught by teaching staff, which later can also provide feedback from the course. Students allow a student to prove that they were enrolled in that course, thus providing feedback. -An example of student representation is: -\begin{verbatim} + An example of student representation is: + \begin{verbatim} Student eefe83c5-555b-4804-a027-182afd6957df University: University 0177545f-2730-496d-8971-6204ff804979 -\end{verbatim} + \end{verbatim} -Students unions track their students ID, allowing to trace the real identity of a student, in case of legal dispute. + Students unions track their student's ID, allowing them to trace the real identity in case of legal dispute. -An example of professor representation is: -\begin{verbatim} + An example of professor representation is: + \begin{verbatim} Professor 43009bef-7c5d-4000-8861-4b2c7805a4e5 University: University 0177545f-2730-496d-8971-6204ff804979 Courses: Course b143ef55-64b7-41b5-b91b-0b8161f98ef6 QUCs: [...]] -\end{verbatim} + \end{verbatim} -QUCs are the list of QUC items that refer that professor. + QUCs are the list of QUC items that refer to that professor. -Hyperledger Fabric distributed ledger contains a world state (value of all objects), and a blockchain that records the history of all transactions that resulted in the current world state. Fabric only accepts a transaction as valid given that it collected enough endorser signatures. + Hyperledger Fabric distributed ledger contains a world state (value of all objects) and a blockchain that records all transactions that resulted in the current world state. Fabric only accepts a transaction as valid, given that it collected enough endorser signatures. -Feedback items are then conceptual objects of value, modeled as states, whose lifecycle transitions are described by transactions. An effective analysis of states and transactions is an essential starting point for a successful implementation. + Feedback items are then conceptual objects of value, modeled as states, whose lifecycle transitions are described by transactions. An effective analysis of states and transactions is an essential starting point for a successful implementation. -We can represent the life cycle of a feedback item using a state transition diagram: + We can represent the life cycle of a feedback item using a state transition diagram: -\begin{figure}[h] - \centering - \includegraphics[scale=0.6]{figures/fig3.pdf} - \caption{State transition diagram for the feedback item lifecycle. States are changed via transactions, issued by different actors.} - \label{fig:feedback_item_lc} -\end{figure} + \begin{figure}[h] + \centering + \includegraphics[scale=0.6]{figures/fig3.pdf} + \caption{State transition diagram for the feedback item lifecycle. States are changed via transactions, issued by different actors.} + \label{fig:feedback_item_lc} + \end{figure} -A feedback item is created by either a student or a professor via a createFeedback transaction, issued via the students union or professors department, respectively. The course is taught during the semester. After it is completed, a fillFeedback transaction can be issued, marking feedback as completed. Finally, when QUC scores are being produced, processFeedback is called, marking the feedback as processed (accounting for producing a QUC score). + A feedback item is created by either a student or a professor via a createFeedback transaction, issued via the students union or professors department, respectively. The course is taught during the semester. After it is completed, a fillFeedback transaction can be issued, marking feedback as completed. Finally, when QUC scores are being produced, processFeedback is called, marking the feedback as processed (accounting for producing a QUC score). -We can represent the life cycle of a course using a state transition diagram: + We can represent the life cycle of a course using a state transition diagram: -\begin{figure}[h!] - \centering - \includegraphics[scale=0.6]{figures/fig4.pdf} - \caption{State transition diagram for the course lifecycle.} - \label{fig:course_lf} -\end{figure} + \begin{figure}[h!] + \centering + \includegraphics[scale=0.6]{figures/fig4.pdf} + \caption{State transition diagram for the course lifecycle.} + \label{fig:course_lf} + \end{figure} -The lifecycle of a course comprises its creation by the university or a professor (createCourse), its normal operation and its finishing (courseCompleted), and its evaluation (evaluateCourse). The university calls evaluateCourse when it is deemed to be finished. At that time, feedback items are processed, and a QUC score is calculated, for both the course and its teaching staff. -The state diagram describes how feedback items and courses change over time, and how specific transactions govern the life cycle transitions. + The lifecycle of a course comprises its creation by the university or a professor (createCourse), its normal operation and its finishing (courseCompleted), and its evaluation (evaluateCourse). The university calls evaluateCourse when it is deemed to be finished. At that time, feedback items are processed, and a QUC score is calculated for both the course and its teaching staff. + The state diagram describes how feedback items and courses change over time and how specific transactions govern the life cycle transitions. -The QUC score is created when after a course is deemed complete, and after the deadline for the QUCs filling is achieved. + The QUC score is created when after a course is deemed complete and after the deadline for the QUCs filling is achieved. -The relationship between all objects stored on the ledger is as follows: + The relationship between all objects stored on the ledger is as follows: -\begin{figure}[h!] - \centering - \includegraphics[scale=0.6]{figures/fig5.pdf} - \caption{Mental model of the concepts involved in B4S QUC} - \label{fig:mental_model} -\end{figure} + \begin{figure}[h!] + \centering + \includegraphics[scale=0.6]{figures/fig5.pdf} + \caption{Mental model of the concepts involved in B4S QUC} + \label{fig:mental_model} + \end{figure} -In Hyperledger Fabric, smart contracts implement transaction logic that transition objects between their different states. The different states of feedback items, courses, etc, are actually held in the ledger world state, as illustrated in the next figure: + In Hyperledger Fabric, smart contracts implement transaction logic that transition objects between their different states. The different states of feedback items, courses, etc., are actually held in the ledger world state, as illustrated in the next figure: -\begin{figure}[h!] - \centering - \includegraphics[scale=0.6]{figures/fig6.pdf} - \caption{A state representing a course is created as a result of a createCourse transaction. State transitions occur as a result of different transactions.} - \label{fig:course_lf_2} -\end{figure} + \begin{figure}[h!] + \centering + \includegraphics[scale=0.6]{figures/fig6.pdf} + \caption{A state representing a course is created as a result of a createCourse transaction. State transitions occur as a result of different transactions.} + \label{fig:course_lf_2} + \end{figure} - -You’re now in a great place to translate these ideas into a smart contract. Don’t worry if your programming is a little rusty, we’ll provide tips and pointers to understand the code. -\subsection{Simplifications} -To simplify our use case, we perform some simplifications: + You’re now in a great place to translate these ideas into a smart contract. Don’t worry if your programming is a little rusty. We’ll provide tips and pointers to understand the code. -\begin{itemize} - \item only consider students' feedback on a course (professors feedback can be implemented in a similar way - \item we assume that for QUC calculation purposes, there is only one parameter ``feedbackParameters'', which represents all the necessary components for feedback - \item we use a simplified network, \emph{test-network}, with two organizations: university and students. This network is based on the test-network from fabric-samples\footnote{https://hyperledger-fabric.readthedocs.io/en/release-2.2/tutorial/commercial\_paper.html} \footnote{https://github.com/hyperledger/fabric-samples}(see Figure \ref{fig:network}. -\end{itemize} + \subsection{Simplifications} + To simplify our use case, we perform some simplifications: -As an exercise, you might complete some parts of this project, market with \texttt{TODO EXERCISE} - -\begin{figure}[h] - \centering - \includegraphics[scale=0.6]{figures/commercial_paper.diagram.testnet.png} - \caption{Our network, compressed of Org1, Org2 and an orderer.} - \label{fig:network} -\end{figure} + \begin{itemize} + \item only consider students' feedback on a course (professors feedback can be implemented in a similar way + \item we assume that for QUC calculation purposes, there is only one parameter ``feedbackParameters'', which represents all the necessary components for feedback + \item we use a simplified network, \emph{test-network}, with two organizations: university and students. This network is based on the test-network from fabric-samples\footnote{https://hyperledger-fabric.readthedocs.io/en/release-2.2/tutorial/commercial\_paper.html} \footnote{https://github.com/hyperledger/fabric-samples}(see Figure \ref{fig:network}. + \end{itemize} + As an exercise, you might complete some parts of this project, market with \texttt{TODO EXERCISE} -\subsection{Smart Contracts \& Chaincode} + \begin{figure}[h] + \centering + \includegraphics[scale=0.6]{figures/commercial_paper.diagram.testnet.png} + \caption{Our network, compressed of Org1, Org2 and an orderer.} + \label{fig:network} + \end{figure} + \subsection{Smart Contracts \& Chaincode} -Smart contracts are the core of a blockchain. They allow to generate new information that is appended to the ledger. Every smart contract has a name that uniquely identifies it within a chaincode. Applications access a particular smart contract within a chaincode using its contract name. Essentially, smart contracts are high level abstractions, encapsulated by chaincode, made available to developers. -A chaincode is a generic container for deploying code to a Hyperledger Fabric blockchain network. It groups smart contracts and can also be used for low level programming of the Fabric network. Thus, smart contracts manage transactions, whereas chaincode governs the management of smart contracts. + + Smart contracts are the core of a blockchain. They allow generating new information that is appended to the ledger. Every smart contract has a name that uniquely identifies it within a chaincode. Applications access a particular smart contract within a chaincode using its contract name. Essentially, smart contracts are high-level abstractions, encapsulated by chaincode, made available to developers. + + A chaincode is a generic container for deploying code to a Hyperledger Fabric blockchain network. It groups smart contracts and can also be used for low-level programming of the Fabric network. Thus, smart contracts manage transactions, whereas chaincode governs the management of smart contracts. %https://hyperledger-fabric.readthedocs.io/en/release-2.2/_images/develop.diagram.20.png @@ -318,67 +318,67 @@ \subsection{Smart Contracts \& Chaincode} %desc haincode A has three smart contracts defined within it, whereas chaincode B has four smart contracts. See how the chaincode name is used to fully qualify a particular smart contract. %mod -In most cases, a chaincode will only have one smart contract defined within it. However, it can make sense to keep related smart contracts together in a single chaincode. + In most cases, a chaincode will only have one smart contract defined within it. However, it can make sense to keep related smart contracts together in a single chaincode. -\subsubsection{Chaincode organization} -In our case we have one chaincode: \emph{T-QUC}. The T-QUC chaincode manages courses, feedback items, an QUCs, and manages the creation, and operations regarding the different actors on the network (universities, students, and professors). + \subsubsection{Chaincode organization} + In our case, we have one chaincode: \emph{T-QUC}. The T-QUC chaincode manages courses, feedback items, a QUCs, and manages the creation and operations regarding the different actors on the network (universities, students, and professors). -The T-QUC chaincode contains several smart contracts: -\begin{itemize} - \item Course + University (Logistics Smart Contract) - \item QUC + Feedback Items (QUC Smart Contract) + The T-QUC chaincode contains several smart contracts: + \begin{itemize} + \item Course + University (Logistics Smart Contract) + \item QUC + Feedback Items (QUC Smart Contract) \item Student + Professor (Users Smart Contract) -\end{itemize} + \end{itemize} -For simplicity, we focus on the students feedback (the professors feedback can be implemented in a very similar fashion). + For simplicity, we focus on the students' feedback (the professors' feedback can be implemented in a very similar fashion). % -\subsubsection{Developing Chaincode} + \subsubsection{Developing Chaincode} + + Chaincode development can be seen as object-oriented programming, with some useful abstractions. Our abstraction regarding the blockchain is implemented through the State and Statelist classes. In short, the blockchain only cares about key-value pairs that compose the blockchain's world state. However, we want to group types of objects (let's say Students, Professors, or Universities) by giving each key a proper identifier (a prefix). The correlated states (let's say the state of Student 1 and Student 2) are further grouped into a Statelist. This allows for easy querying of the world state. -Chaincode development can be seen as object-oriented programming, with some useful abstractions. Our abstraction regarding the blockchain is implemented through the State and Statelist classes. In short, the blockchain only cares about key-value pairs - that composes the world state of the blockchain. However, we want to group types of objects (lets say Students, Professors, or Universities), by giving each key a proper identifier (a prefix). The correlated states (lets say the state of Student 1 and Student 2) are further grouped into a Statelist. This allows easy querying of the world state. + Open \emph{state.js}, inside the directory \emph{ledger-api} with your favorite editor. A state is just a reference to a key on the blockchain that holds a certain value. The key might be simple (single value) or composed (a combination of values, separated by ``:''). We support composed keys, supported by the \emph{makeKey} method: + \begin{minted}[breaklines]{javascript} -Open \emph{state.js}, inside the directory \emph{ledger-api} with your favorite editor. A state is just a reference to a key on the blockchain that holds a certain value. The key might be simple (single value), or composed (a combination of values, separated by ``:''). We support composed keys, supported by the \emph{makeKey} method: -\begin{minted}[breaklines]{javascript} - this.key = State.makeKey(keyParts); - + ... - + static makeKey(keyParts) { - return keyParts.map(part => JSON.stringify(part)).join(':'); + return keyParts.map(part => JSON.stringify(part)).join(':'); } -\end{minted} + \end{minted} -Furthermore, the State class provides useful methods for serializing and deserializing the information. In other words, the information is passed to the ledger in the JSON format (the value corresponding to a key). It is desirable that when we query or retrieve information from the ledger we obtain an object from a certain class, not plain JSON. + Furthermore, the State class provides useful methods for serializing and deserializing the information. In other words, the information is passed to the ledger in the JSON format (the value corresponding to a key). It is desirable that when we query or retrieve information from the ledger, we obtain an object from a certain class, not plain JSON. -Now, focus on \emph{statelist.js}. Here is where the magic happens: 1) adding information to the blockchain; 2) updating a state; 3) obtaining a state; 4) querying the blockchain. + Now, focus on \emph{statelist.js}. Here is where the magic happens: 1) adding information to the blockchain; 2) updating a state; 3) obtaining a state; 4) querying the blockchain. -For instance, the following method adds a new state to the blockchain. Note that this method is exactly equal to \emph{updateState}, because the primitive \emph{putState} adds or updates a state, if the state does not or does already exist, respectively. It is desirable to differentiate them because they are semantically different. Basically, to add a new state, we retrieve the key stored in the State, serialize the data, and call the blockchain API. + For instance, the following method adds a new state to the blockchain. Note that this method is exactly equal to \emph{updateState}, because the primitive \emph{putState} adds or updates a state if the state does not or does already exist, respectively. It is desirable to differentiate them because they are semantically different. Basically, to add a new state, we retrieve the key stored in the State, serialize the data, and call the blockchain API. -\begin{minted}[breaklines]{javascript} - async addState(state) { + \begin{minted}[breaklines]{javascript} + async addState(state) { let key = this.ctx.stub.createCompositeKey(this.name, state.getSplitKey()); let data = State.serialize(state); await this.ctx.stub.putState(key, data); - } + } -\end{minted} + \end{minted} -Now, the existing classes (Course, University, QUC, Feedback, Professor, Student) are just specialized states, with their own attributes and methods. For instance, the Course has methods to change an attribute called current state, which can be either Created, Completed, or Evaluated (see Figure \ref{fig:course_lf_2}): -\begin{minted}[breaklines]{javascript} -const State = require('../../ledger-api/state.js'); + Now, the existing classes (Course, University, QUC, Feedback, Professor, Student) are just specialized states, with their own attributes and methods. For instance, the Course has methods to change an attribute called current state, which can be either Created, Completed, or Evaluated (see Figure \ref{fig:course_lf_2}): + \begin{minted}[breaklines]{javascript} + const State = require('../../ledger-api/state.js'); -const courseState = { - CREATED: 0, - COMPLETED: 1, - EVALUATED: 2 -}; + const courseState = { + CREATED: 0, + COMPLETED: 1, + EVALUATED: 2 + }; -//course item -class Course extends State { + //course item + class Course extends State { - constructor(obj) { + constructor(obj) { //Creator ID corresponds to the university ID super(Course.getClass(), [obj.universityId, obj.courseId]); this.currentState = courseState.CREATED; @@ -386,238 +386,235 @@ \subsubsection{Developing Chaincode} this.teachingStaff = obj.teachingStaff; this. students = obj.students; Object.assign(this, obj); - } + } - setStateToCompleted() { + setStateToCompleted() { this.currentState = courseState.COMPLETED; - } - - [...] + } + + [...] static fromBuffer(buffer) { return course.deserialize(buffer); - } + } - toBuffer() { + toBuffer() { return Buffer.from(JSON.stringify(this)); - } + } - static deserialize(data) { + static deserialize(data) { return State.deserializeClass(data, Course); - } + } - /** - * Factory method to create a course Item - */ - static createInstance(courseId, universityId, teachingStaff, students) { + /** + * Factory method to create a course Item + */ + static createInstance(courseId, universityId, teachingStaff, students) { return new Course({ courseId, universityId, teachingStaff, students }); - } + } - static getClass() { + static getClass() { return "org.b4s.course"; - } + } -\end{minted} + \end{minted} -Now, the other crucial piece is the corresponding lists: in the course case, the course-list. Whereas a course is a state, a course-list is a state-list. In the list, we are defining the logic of interacting with the chain, something like a middleware layer in the backend of a centralized webapp. For example, the course-list contains a method \emph{getAllCourses}, which uses the \emph{getQueryResults} method from the state-list: -\begin{minted}[breaklines]{javascript} - async getAllCourses() { - let query = { - "selector": { - "class": "org.b4s.course" - } - }; + Now, the other crucial piece is the corresponding lists: in the course case, the course-list. Whereas a course is a state, a course-list is a state-list. In the list, we are defining the logic of interacting with the chain, something like a middleware layer in the backend of a centralized webapp. For example, the course-list contains a method \emph{getAllCourses}, which uses the \emph{getQueryResults} method from the state-list: + \begin{minted}[breaklines]{javascript} + async getAllCourses() { + let query = { + "selector": { + "class": "org.b4s.course" + } + }; - let prepared_query = JSON.stringify(query); - return await this.getQueryResults(prepared_query); - } + let prepared_query = JSON.stringify(query); + return await this.getQueryResults(prepared_query); + } -\end{minted} + \end{minted} -Finally, we develop our smart contracts on top of these objects + Finally, we develop our smart contracts on top of these objects %aquii -For more information, refer to the commercial paper example\footnote{https://hyperledger-fabric.readthedocs.io/en/release-2.2/tutorial/commercial\_paper.html}, as well as fabric-samples\footnote{https://github.com/hyperledger/fabric-samples}. + For more information, refer to the commercial paper example\footnote{https://hyperledger-fabric.readthedocs.io/en/release-2.2/tutorial/commercial\_paper.html}, as well as fabric-samples\footnote{https://github.com/hyperledger/fabric-samples}. -\subsubsection{Wrap up} -Confused? Do not worry. A blockchain is a complex system. For leveraging a blockchain system, an equilibrium between decentralized and centralized components has to be achieved. We can envision B4S as a full-stack system, in which the database is a blockchain. A user interface communicates with a blockchain client, which is a server. This server can act on behalf of an organization, issuing transactions against a peer node, that redirects it to a chaincode container. This container holds several smart contracts (user, logistics, quc). Each smart contract is divided into the basic entities (User, Feedback), and useful abstractions that help us developing chaincode (State, Statelist). This is illustrated in Figure \ref{fig:b4s}. + \subsubsection{Wrap up} + Confused? Do not worry. A blockchain is a complex system. For leveraging a blockchain system, an equilibrium between decentralized and centralized components has to be achieved. We can envision B4S as a full-stack system, in which the database is a blockchain. A user interface communicates with a blockchain client, which is a server. This server can act on behalf of an organization, issuing transactions against a peer node that redirects it to a chaincode container. This container holds several smart contracts (user, logistics, quc). Each smart contract is divided into the basic entities (User, Feedback), and useful abstractions help us develop chaincode (State, Statelist). This is illustrated in Figure \ref{fig:b4s}. -\begin{figure}[h] - \centering - \includegraphics[scale=0.6]{figures/b4s_architecture.pdf} - \caption{B4S architecture} - \label{fig:b4s} -\end{figure} + \begin{figure}[h] + \centering + \includegraphics[scale=0.6]{figures/b4s_architecture.pdf} + \caption{B4S architecture} + \label{fig:b4s} + \end{figure} -\section{Setting up B4S} -\subsection{Install prerequisites} -This project has several prerequisites. Please refer to the \emph{README.md} for instructions. + \section{Setting up B4S} + \subsection{Install prerequisites} + This project has several prerequisites. Please refer to the \emph{README.md} for instructions. -For a straightforward setup, clone the repository at \texttt{https://github.com/hyperledger-labs/university-course/tree/master/support/Lab05}. Go to the B4S folder, by running \texttt{cd b4s}. Install the prerequesites by running \texttt{./install-prerequesites}. In case of any problem, refer to the \emph{Troubleshooting} section of the \emph{README.md}. + For a straightforward setup, clone the repository at \texttt{https://github.com/hyperledger-labs/university-course/tree/master/support/Lab05}. Go to the B4S folder by running \texttt{cd b4s}. Install the prerequisites by running \texttt{./install-prerequesites}. In case of any problem, refer to the \emph{Troubleshooting} section of the \emph{README.md}. -All issues should be reported at: https://github.com/hyperledger-labs/university-course/issues, with the label \emph{lab-question}, and an extra lab referring to this lab number. + All issues should be reported at https://github.com/hyperledger-labs/university-course/issues, with the label \emph{lab-question}, and an extra lab referring to this lab number. -\subsection{Instantiate the network} -On the b4s directory, run \texttt{./network-starter}. This script calls \emph{network} and \emph{deployCC} scripts, present on the \emph{test-network} directories. Do not worry about the details of these scripts. We will cover infrastructure on the next lab. + \subsection{Instantiate the network} + On the b4s directory, run \texttt{./network-starter}. This script calls \emph{network} and \emph{deployCC} scripts, present on the \emph{test-network} directories. Do not worry about the details of these scripts. We will cover the infrastructure in the next lab. -The execution should return no errors. After it is completed, run \emph{docker ps}. Notice that it created a network composed of 2 organizations, 2 peers, 1 orderer, 2 CAs, and 2 couchdb databases. + The execution should return no errors. After it is completed, run \emph{docker ps}. Notice that it created a network composed of 2 organizations, 2 peers, 1 orderer, 2 CAs, and 2 CouchDB databases. -\begin{figure}[h] - \centering - \includegraphics[scale=0.08]{figures/output_docker_ps.png} - \caption{Result of running the \emph{docker ps} command, showing the created infrastructure} - \label{fig:docker} -\end{figure} + \begin{figure}[h] + \centering + \includegraphics[scale=0.08]{figures/output_docker_ps.png} + \caption{Result of running the \emph{docker ps} command, showing the created infrastructure} + \label{fig:docker} + \end{figure} -\subsection{Interact with the network} -Congratulations! You have a working Hyperledger Fabric v2.2 blockchain running on your computer. Now, we shall interact with it. + \subsection{Interact with the network} + Congratulations! You have a working Hyperledger Fabric v2.2 blockchain running on your computer. Now, we shall interact with it. -Start by examining the first chaincode container, by running \emph{docker logs CONTAINER\_ID}. In my case, I should run \emph{docker logs 1a336c331f4a}, as illustrated in Figure \ref{fig:docker}. + Start by examining the first chaincode container, by running \emph{docker logs CONTAINER\_ID}. In my case, I should run \emph{docker logs 1a336c331f4a}, as illustrated in Figure \ref{fig:docker}. -You shall obtain a similar output: + You shall obtain a similar output: -\begin{figure}[h] - \centering - \includegraphics[scale=0.08]{figures/docker_logs_chaincode.png} - \caption{Result of running the \emph{docker logs 1a336c331f4a} command.} - \label{fig:docker_chaincode} -\end{figure} + \begin{figure}[h] + \centering + \includegraphics[scale=0.08]{figures/docker_logs_chaincode.png} + \caption{Result of running the \emph{docker logs 1a336c331f4a} command.} + \label{fig:docker_chaincode} + \end{figure} %To interact with the network as the university entity, go to \texttt{Lab05/b4s/organization/university}, and run \texttt{./university.sh}. This loads the necessary environment variables to interact with the corresponding peer that belongs to the university organization (organization 2). Conversely, to interact with the network as the student entity, go to \texttt{Lab05/b4s/organization/students-union}, and run \texttt{./students.sh}. This loads the necessary environment variables to interact with the corresponding peer that belongs to organization 1. -To interact with the students-union peer, you need to interact with peer0 from organization 1, whereas if you want to interact with the university peer, you need to interact with peer0 from organization 2. -To get information about the channel in which the peers are connected, run \texttt{docker exec peer0.org1.example.com peer channel getinfo -c mychannel}. - -Another way is to load the configuration environment variables. To do this, from the \emph{b4s} folder, run (also see section \emph{Interacting with the network}, from the README.md: + To interact with the students-union peer, you need to interact with peer0 from organization 1. In contrast, if you want to interact with the university peer, you need to interact with peer0 from organization 2. + To get information about the channel in which the peers are connected, run \texttt{docker exec peer0.org1.example.com peer channel getinfo -c mychannel}. -\begin{minted}{bash} -source organization/university/university.sh -\end{minted} + Another way is to load the configuration environment variables. To do this, from the \emph{b4s} folder, run (also see section \emph{Interacting with the network}, from the README.md: -Right now, you can issue commands on behalf of the university peer. + \begin{minted}{bash} + source organization/university/university.sh + \end{minted} -\subsection{Issuing Transactions} + Right now, you can issue commands on behalf of the university peer. -You can see the result of the execution by analyzing the docker container logs: \emph{docker logs 1a336c331f4a}, and \emph{docker logs peer0.org1.example.com}. + \subsection{Issuing Transactions} -Let us issue a transaction against the QUC smart contract. For that, we need information regarding our certificates, in which the ABSOLUTE\_PATH variable is the absolute path on your local machine to the lab (careful copying the code, because the buffer might contain invalid characters: + You can see the result of the execution by analyzing the docker container logs: \emph{docker logs 1a336c331f4a}, and \emph{docker logs peer0.org1.example.com}. + Let us issue a transaction against the QUC smart contract. For that, we need information regarding our certificates, in which the ABSOLUTE\_PATH variable is the absolute path on your local machine to the lab (careful copying the code, because the buffer might contain invalid characters: -\begin{minted}[ -breaklines, -breakafter=test-network - mathescape, - linenos, - numbersep=5pt, - numbersep=5pt, - xleftmargin=0pt]{bash} -peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile ABSOLUTE_PATH/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n b4s --peerAddresses localhost:7051 --tlsRootCertFiles ABSOLUTE_PATH/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles ABSOLUTE_PATH/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"function":"org.b4s.quc:start", "Args":[]}' --waitForEvent -\end{minted} -For instance, the full command would be: -\begin{minted}[ -breaklines, -breakafter=test-network - mathescape, - linenos, - numbersep=5pt, - numbersep=5pt, - xleftmargin=0pt]{bash} -peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n b4s --peerAddresses localhost:7051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"function":"org.b4s.quc:start", "Args":[]}' --waitForEvent -\end{minted} + \begin{minted}[ + breaklines, + breakafter=test-network + mathescape, + linenos, + numbersep=5pt, + numbersep=5pt, + xleftmargin=0pt]{bash} + peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile ABSOLUTE_PATH/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n b4s --peerAddresses localhost:7051 --tlsRootCertFiles ABSOLUTE_PATH/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles ABSOLUTE_PATH/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"function":"org.b4s.quc:start", "Args":[]}' --waitForEvent + \end{minted} -Try, and issue another transaction: -\begin{minted}[ -breaklines, -breakafter=test-network - mathescape, - linenos, - numbersep=5pt, - numbersep=5pt, - xleftmargin=0pt]{bash} -peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n b4s --peerAddresses localhost:7051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"function":"org.b4s.quc:showIdentity", "Args":[]}' --waitForEvent -\end{minted} + For instance, the full command would be: + \begin{minted}[ + breaklines, + breakafter=test-network + mathescape, + linenos, + numbersep=5pt, + numbersep=5pt, + xleftmargin=0pt]{bash} + peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n b4s --peerAddresses localhost:7051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"function":"org.b4s.quc:start", "Args":[]}' --waitForEvent + \end{minted} -What did you notice? This transaction obtains and returns information about the party calling the transaction. Inspect the logs: \emph{docker logs 1a336c331f4a}. You can now see that the transaction was processed successfully. + Try, and issue another transaction: + \begin{minted}[ + breaklines, + breakafter=test-network + mathescape, + linenos, + numbersep=5pt, + numbersep=5pt, + xleftmargin=0pt]{bash} + peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n b4s --peerAddresses localhost:7051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"function":"org.b4s.quc:showIdentity", "Args":[]}' --waitForEvent + \end{minted} -\begin{figure}[h] - \centering - \includegraphics[scale=0.08]{figures/chaincode_response_1.png} - \caption{Logs of one of the chaincode containers, after issuing \emph{showIdentity}} - \label{fig:cr_1} -\end{figure} + What did you notice? This transaction obtains and returns information about the party calling the transaction. Inspect the logs: \emph{docker logs 1a336c331f4a}. You can now see that the transaction was processed successfully. -Finally, let us create a university! Run: -\begin{minted}[ -breaklines, -breakafter=test-network - mathescape, - linenos, - numbersep=5pt, - numbersep=5pt, - xleftmargin=0pt]{bash} -peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n b4s --peerAddresses localhost:7051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"function":"org.b4s.logistics:createUniversity", "Args":["tecnicoLisboa", "Portugal"]}' --waitForEvent -\end{minted} + \begin{figure}[h] + \centering + \includegraphics[scale=0.08]{figures/chaincode_response_1.png} + \caption{Logs of one of the chaincode containers, after issuing \emph{showIdentity}} + \label{fig:cr_1} + \end{figure} -Observe the output and the chaincode logs. -You just created your first asset on a blockchain. You can check it on the CouchDB user interface, which holds the world state. To access it, on your browser, go to: -\begin{minted}{bash} -http://localhost:5984/_utils/#database/mychannel_b4s/_all_docs -\end{minted} + Finally, let us create a university! Run: + \begin{minted}[ + breaklines, + breakafter=test-network + mathescape, + linenos, + numbersep=5pt, + numbersep=5pt, + xleftmargin=0pt]{bash} + peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n b4s --peerAddresses localhost:7051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles /home/rafael/Projects/university-course/support/Lab05/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"function":"org.b4s.logistics:createUniversity", "Args":["tecnicoLisboa", "Portugal"]}' --waitForEvent + \end{minted} -You should see the created object: + Observe the output and the chaincode logs. + You just created your first asset on a blockchain. You can check it on the CouchDB user interface, which holds the world state. To access it, on your browser, go to: + \begin{minted}{bash} + http://localhost:5984/_utils/#database/mychannel_b4s/_all_docs + \end{minted} -\begin{figure}[h] - \centering - \includegraphics[scale=0.2]{figures/couchdb.png} - \caption{CouchDB user interface, b4s channel} - \label{fig:cdb} -\end{figure} + You should see the created object: -\subsection{Clean Up} + \begin{figure}[h] + \centering + \includegraphics[scale=0.2]{figures/couchdb.png} + \caption{CouchDB user interface, b4s channel} + \label{fig:cdb} + \end{figure} -In the next class, we will explore in more detail the network topology, operations (start the network, monitor the network), the smart contract lifecycle in Fabric v2.x, and off-chain blockchain components (this is, the blockchain client application and user interfaces that allows end-users to interact with the blockchain). This will help you understand Fabric as a whole. + \subsection{Clean Up} -Do not forget the network, which is running and consuming resources! To delete the network, go to the b4s folder, and run: \texttt{./network-clean}. + In the next class, we will explore in more detail the network topology, operations (start the network, monitor the network), the smart contract lifecycle in Fabric v2.x, and off-chain blockchain components (this is, the blockchain client application and user interfaces that allows end-users to interact with the blockchain). This will help you understand Fabric as a whole. + Do not forget the network, which is running and consuming resources! To delete the network, go to the b4s folder, and run \texttt{./network-clean}. -\section{Exercises} + \section{Exercises} -%How to add the students grades in order to not only provide reputation on answering the QUCs, but also on their grades (like Qualichain). -\subsubsection*{How to provide extra differentiation for teaching quality?} + \subsubsection*{How to provide extra differentiation for teaching quality?} -Different levels of excellence awards (e.g. quc $\geq$ 8, quc $\geq$ 8.5; quc = 9 -\subsubsection*{One of the critiques to QUCs is that a professors score depends on his or her popularity. How to decrease this risk?} + \subsubsection*{One of the critiques to QUCs is that a professor's score depends on his or her popularity. How to decrease this risk?} -\subsubsection*{What is the scalability of B4S QUC, given that Hyperledger Fabric can only contain a limited number of peer nodes and orgs?} + \subsubsection*{What is the scalability of B4S QUC, given that Hyperledger Fabric can only contain a limited number of peer nodes and orgs?} -\subsubsection*{When is a transaction considered valid in this ecosystem?} + \subsubsection*{When is a transaction considered valid in this ecosystem?} -\subsubsection*{Imagine that the T-QUC chaincode is split into T-QUC Management and Actor Management chaincodes. Wwhat are the pros and cons of this approach?} + \subsubsection*{Suppose that the T-QUC chaincode is split into T-QUC Management and Actor Management chaincodes. What are the pros and cons of this approach?} -\subsubsection*{To get information about the channel in which the peers are connected, run {docker exec peer0.org1.example.com peer channel getinfo -c mychannel}. What do you see?} + \subsubsection*{To get information about the channel in which the peers are connected, run {docker exec peer0.org1.example.com peer channel getinfo -c mychannel}. What do you see?} -\subsubsection*{Analyze the chaincode container logs, as well as the peer logs. What do you see?} + \subsubsection*{Analyze the chaincode container logs, as well as the peer logs. What do you see?} -\subsubsection*{Instead of the chaincode receiving a unique Id as a parameter, we could generate it on the smart contract itself (e.g., let uniqueId = uuidv4()). This way, the issuer may not tamper or control the generated IDs. Why is this a bad idea?} + \subsubsection*{Instead of the chaincode receiving a unique Id as a parameter, we could generate it on the smart contract itself (e.g., let uniqueId = uuidv4()). This way, the issuer may not tamper or control the generated IDs. Why is this a bad idea?} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT ENDS HERE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\bibliographystyle{abbrv} -\bibliography{lab.bib} + \bibliographystyle{abbrv} + \bibliography{lab.bib} \end{document} % The required last line \ No newline at end of file diff --git a/source/Lab05/Instructors Guide/assignment.tex b/source/Lab05/Instructors Guide/assignment.tex index d117da3..0d048bd 100644 --- a/source/Lab05/Instructors Guide/assignment.tex +++ b/source/Lab05/Instructors Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -19,8 +19,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -70,16 +70,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 5\\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module II - Hyperledger Fabric} & \textbf{Issue Date:} & - \\ +\multicolumn{2}{|l|}{Module II - Hyperledger Fabric} & \textbf{Issue Date:} & - \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Hyperledger Fabric: Infrastructure and Chaincode} & \textbf{Due Date:} & - \\ +\multicolumn{2}{|l|}{Hyperledger Fabric: Infrastructure and Chaincode} & \textbf{Due Date:} & - \\ \hline \end{tabularx}} \rhead{} @@ -96,42 +96,41 @@ \section*{Instructors Guide} -The majority of this laboratory was inspired on the implementation of JusticeChain\cite{belchior2019_audits}. +The majority of this laboratory was inspired by the implementation of JusticeChain\cite{belchior2019_audits}. \subsubsection*{How to provide extra differentiation for teaching quality?} -Different levels of excellence awrads (e.g. quc $\geq$ 8, quc $\geq$ 8.5; quc = 9 +Different levels of excellence awards (e.g., quc $\geq$ 8, quc $\geq$ 8.5; quc = 9 -\subsubsection*{One of the critiques to QUCs is that a professors score depends on his or her popularity. How to decrease this risk?} +\subsubsection*{One of the critiques to QUCs is that a professor's score depends on his or her popularity. How to decrease this risk?} -Have a set of questions directed to ascertain a professors popularity (eg. how much does the professor care about the students; how does the prof empatise with the class, etc). If a student rates a professors popularity too high, his QUC score might need to be normalized. The same if the popularity is too low. +Have a set of questions directed to ascertain a professor's popularity (e.g., how much the professor cares about the students; how does the professor empathize with the class, etc.). If a student rates a professor's popularity too high, his QUC score might need to be normalized. The same if the popularity is too low. \subsubsection*{What is the scalability of B4S QUC, given that Hyperledger Fabric can only contain a limited number of peer nodes and orgs?} -A: Recent experiments show that Fabric can accomodate 32 organizations, in a total of 128 peer nodes\footnote{https://www.ibm.com/blogs/blockchain/2019/04/does-hyperledger-fabric-perform-at-scale/}. If each orgnaization represents an university ecosystem, and each university has 3 peer nodes (university, professor, student), then at least 32 universities can be fit into a small system. +A: Recent experiments show that Fabric can accommodate 32 organizations, in a total of 128 peer nodes\footnote{https://www.ibm.com/blogs/blockchain/2019/04/does-hyperledger-fabric-perform-at-scale/}. If each organization represents a university ecosystem, and each university has 3 peer nodes (university, professor, student), then at least 32 universities can be fit into a small system. \subsubsection*{When is a transaction considered valid in this ecosystem?} -A: Fabric only accepts a transaction as valid given that it collected enough endorser signatures (as stipulated in the endorsement policy). Common policies are ALL or at least 1. - +A: Fabric only accepts a transaction as valid, given that it collected enough endorser signatures (as stipulated in the endorsement policy). Common policies are ALL or at least 1. -\subsubsection*{Imagine that the T-QUC chaincode is split into T-QUC Management and Actor Management chaincodes. Wwhat are the pros and cons of this approach?} +\subsubsection*{Suppose that the T-QUC chaincode is split into T-QUC Management and Actor Management chaincodes. What are the pros and cons of this approach?} - Pros: better chaincode management, by trying to isolate classes not directly related (universities, professors and students do not depend on the QUCs). However, as the data is losely related, there are issues we need to be aware. +Pros: better chaincode management by isolating classes not directly related (universities, professors, and students do not depend on the QUCs). However, as the data is loosely related, there are issues we need to be aware of. Cons: The need to manage two chaincodes in terms of operations and chaincode development. In fact, ``if a smart contract needs to access another chaincode world state, it can do this by performing a chaincode-to-chaincode invocation`''. \footnote{https://hyperledger-fabric.readthedocs.io/en/release-2.2/developapps/chaincodenamespace.html}. Control is passed between chaincode using the invokeChaincode() API. \subsubsection*{To get information about the channel in which the peers are connected, run {docker exec peer0.org1.example.com peer channel getinfo -c mychannel}. What do you see?} A: Blockchain info: {"height":6,"currentBlockHash":"hiLTR1dF+bk3EkUNusE7q1pkh939cQzu5pgQrUlrK1M=","previousBlockHash":"iHUt4lRf5wMJa/O70xuRbGRT2RG3RvdvWXGW8s0j9Uc="} -Current block number, current block hash and previous block hash. +Containing: current block number, current block hash, and previous block hash. \subsubsection*{Analyze the chaincode container logs, as well as the peer logs. What do you see?} A: You can see the result of the execution by analyzing the docker container logs: \emph{docker logs 1a336c331f4a}, and \emph{docker logs peer0.org1.example.com}. -Peer: operations regarding the orderer, and endorsement of transactions, as well as receiving requests to execute chaincode. The chaincode container contains the logs of the chaincode (logs errors and console.log()). The peer returns if the invocation was successful or not. +Peer: operations regarding the orderer and endorsement of transactions and receiving requests to execute chaincode. The chaincode container contains the logs of the chaincode (logs errors and console.log()). The peer returns if the invocation was successful or not. -\subsubsection*{Instead of the chaincode receiving a unique Id as a parameter, we could generate it on the smart contract itself (e.g., let uniqueId = uuidv4()). This way, the issuer may not tamper or control the generated IDs. Why is this a bad idea?} +\subsubsection*{Instead of the chaincode receiving a unique Id as a parameter, we could generate it on the smart contract itself (e.g., let uniqueId = uuidv4()). This way, the issuer may not tamper with or control the generated IDs. Why is this a bad idea?} A: You will obtain the error \emph{Error: could not assemble transaction: ProposalResponsePayloads do not match} because all peers are endorsing the transaction. As the uuidv4() method is not deterministic, each peer generates a different ID. This causes a mismatch that does not respect the endorsement policy -- leading to the transaction failure. diff --git a/source/Lab06/Guide/assignment.tex b/source/Lab06/Guide/assignment.tex index d1afebc..aa2a9bf 100644 --- a/source/Lab06/Guide/assignment.tex +++ b/source/Lab06/Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -28,8 +28,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -79,16 +79,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 6 \\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module II - Hyperledger Fabric} & \textbf{Issue Date:} & - \\ +\multicolumn{2}{|l|}{Module II - Hyperledger Fabric} & \textbf{Issue Date:} & - \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Hyperledger Fabric: Full-Stack dApp} & \textbf{Due Date:} & - \\ +\multicolumn{2}{|l|}{Hyperledger Fabric: Full-Stack dApp} & \textbf{Due Date:} & - \\ \hline \end{tabularx}} \rhead{}} @@ -104,53 +104,52 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 \section*{Preliminary Notes} -The majority of this laboratory was inspired on the implementation of JusticeChain\cite{belchior2019_audits}. Part of this laboratory was inspired in sources elaborated by KC Tam\footnote{https://medium.com/@kctheservant}. +The majority of this laboratory was inspired by the implementation of JusticeChain\cite{belchior2019_audits}. Part of this laboratory was inspired in sources elaborated by KC Tam\footnote{https://medium.com/@kctheservant}. -Working on the blockchain ecosystem includes two major components: smart contract (chaincode) development, and operations (infrastructure). We will learn how to create the chaincode to implement the system B4S QUC. +Working on the blockchain ecosystem includes two major components: smart contract (chaincode) development and operations (infrastructure). We will learn how to create the chaincode to implement the system B4S QUC. Contrarily to previous laboratories, this laboratory does not have exercises at the end of this guide. Instead, you will take a look at the code and experiment with our system while you navigate through this guide. -To do so, please clone the repository, using the command: -{git clone https://github.com/hyperledger-labs/university-course} +To do so, please clone the repository at, using the command: +{git clone} %The reference section should be viewed as a ``additional readings reference'' - if you would like more information about the topic. +This laboratory uses the service \emph{carbon.sh} to create some figures. -This lab uses carbon.sh to create some figures. - -You will learn: +You will learn \begin{enumerate} - \item What is the process behind creating a blockchain network - \item How does a web app related to the blockchain - \item An example of a complete, full-stack decentralized application running Fabric 2.2 +\item What is the process behind creating a blockchain network +\item How does a web app related to the blockchain +\item An example of a complete, full-stack decentralized application running Fabric 2.2 \end{enumerate} Let us recall the architecture of our application: \begin{figure}[h] - \centering - \includegraphics[scale=0.6]{figures/b4s_architecture.pdf} - \caption{B4S architecture} - \label{fig:b4s} +\centering +\includegraphics[scale=0.6]{figures/b4s_architecture.pdf} +\caption{B4S architecture} +\label{fig:b4s} \end{figure} -As discussed on the last lab, we can envision B4S as a full-stack system, in which the database is a blockchain. A user interface communicates with a blockchain client, which is a server. This server can act on behalf of an organization, issuing transactions against a peer node, that redirects it to a chaincode container. This container holds several smart contracts (user, logistics, quc), which are divided into the basic entities (User, Feedback) and useful abstractions that help us develop chaincode (State, Statelist). This is illustrated in Figure \ref{fig:b4s}. +As discussed in the last lab, we can envision B4S as a full-stack system, in which the database is a blockchain. A user interface communicates with a blockchain client, which is a server. This server can act on behalf of an organization, issuing transactions against a peer node that redirects it to a chaincode container. This container holds several smart contracts (user, logistics, quc). Each smart contract is divided into the basic entities (User, Feedback), and useful abstractions help us develop chaincode (State, Statelist). This is illustrated in Figure \ref{fig:b4s}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT STARTS HERE -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Blockchain Network} -Our network is composed by 2 organizations, each one with one Peer, one CA and one CouchDB instance. A channel mychannel (in our case, it is called b4s) connects all components. One Orderer orders transactions of the network, as illustrated in Figure \ref{fig:architecture}. +Our network is composed by 2 organizations, each one with one peers, one CA, one couchDB instance. A channel mychannel (in our case, it is called b4s) connects all components. One orderer orders transactions of the network, as illustrated by the next Figure: \begin{figure}[h] - \centering - \includegraphics[scale=0.6]{figures/network.png} - \caption{Network Architecture. Source: https://medium.com/@kctheservant/add-a-peer-to-an-organization-in-test-network-hyperledger-fabric-v2-2-4a08cb901c98} - \label{fig:architecture} +\centering +\includegraphics[scale=0.6]{figures/network.png} +\caption{Network Architecture. Source: https://medium.com/@kctheservant/add-a-peer-to-an-organization-in-test-network-hyperledger-fabric-v2-2-4a08cb901c98} +\label{fig:architecture} \end{figure} Let us see in more detail what happens at the \texttt{network-starter.sh}, and \texttt{network.sh} scripts. @@ -158,10 +157,10 @@ \section{Blockchain Network} \subsection{Deploying the network} \begin{figure}[h] - \centering - \includegraphics[scale=0.4]{figures/steps.png} - \caption{Network starter script steps. Source: https://medium.com/@kctheservant/add-a-peer-to-an-organization-in-test-network-hyperledger-fabric-v2-2-4a08cb901c98} - \label{fig:steps} +\centering +\includegraphics[scale=0.4]{figures/steps.png} +\caption{Network starter script steps. Source: https://medium.com/@kctheservant/add-a-peer-to-an-organization-in-test-network-hyperledger-fabric-v2-2-4a08cb901c98} +\label{fig:steps} \end{figure} On the B4S folder, when the \texttt{network-starter.sh} script is called, it performs some cleanup, and calls the \texttt{network.sh} script twice, located on the \texttt{test-network} folder. The first call is to instantiate the network, while the second calls installs the chaincode: @@ -177,77 +176,75 @@ \subsection{Deploying the network} ./network.sh deployCC -l javascript \end{minted} -Figure \ref{fig:steps} denote the necessary steps to start the network. Please open the \texttt{network.sh} script for inspection. As the flags given to the script are \texttt{up} and \texttt{createChannel}, the functions \texttt{networkUp} and \texttt{createChannel} are called. +Figure \ref{fig:steps} denote the necessary steps to start the network. Please open the \texttt{network.sh} script for inspection. As the flags given to the script are up and createChannel, the functions networkUp, and createChannel are called. -Firstly, the necessary cryptographic material has to be generated, in one of two ways: cryptogen or Fabric CA. Fabric CA is the best way to generate the necessary files, as it is tailored to be used in production. It allows us to generate new components or new users for an organization. This is achieved in the \texttt{registerEnroll.sh} script, and then the \texttt{createOrgs} function: +Firstly, the necessary cryptographic material has to be generated, in one of two ways: cryptogen or Fabric CA. Fabric CA is the best way to generate the necessary files, as it is tailored to be used in production. It allows to generate new components or new users for an organization. This is achieved in the \texttt{registerEnroll.sh} script, and then the createOrgs function: \begin{minted}{bash} echo "##########################################################" - echo "##### Generate certificates using Fabric CA's ############" - echo "##########################################################" +echo "##### Generate certificates using Fabric CA's ############" +echo "##########################################################" - IMAGE_TAG=$IMAGETAG_CA docker-compose -f $COMPOSE_FILE_CA up -d 2>&1 +IMAGE_TAG=$IMAGETAG_CA docker-compose -f $COMPOSE_FILE_CA up -d 2>&1 - . organizations/fabric-ca/registerEnroll.sh +. organizations/fabric-ca/registerEnroll.sh - sleep 10 +sleep 10 - echo "##########################################################" - echo "############ Create Org1 Identities ######################" - echo "##########################################################" +echo "##########################################################" +echo "############ Create Org1 Identities ######################" +echo "##########################################################" - createOrg1 +createOrg1 - echo "##########################################################" - echo "############ Create Org2 Identities ######################" - echo "##########################################################" +echo "##########################################################" +echo "############ Create Org2 Identities ######################" +echo "##########################################################" - createOrg2 +createOrg2 - echo "##########################################################" - echo "############ Create Orderer Org Identities ###############" - echo "##########################################################" +echo "##########################################################" +echo "############ Create Orderer Org Identities ###############" +echo "##########################################################" - createOrderer +createOrderer - fi +fi - echo - echo "Generate CCP files for Org1 and Org2" - ./organizations/ccp-generate.sh +echo +echo "Generate CCP files for Org1 and Org2" +./organizations/ccp-generate.sh \end{minted} -For example, to register the students union president, on behalf of org1, the \texttt{registerEnroll} script calls the Fabric CA client: +For example, to register the students union president, on behalf of org1, the registerEnroll script calls the Fabric CA client: \begin{minted}[breaklines]{bash} - echo "================ ORG1 IDENTITIES ==========================" +echo "================ ORG1 IDENTITIES ==========================" - echo "## REGISTER studentUnionPresident" +echo "## REGISTER studentUnionPresident" fabric-ca-client register -d --id.name studentUnionPresident --id.affiliation org1 --id.attrs '"hf.Registrar.Roles=peer,client"' --id.attrs hf.Revoker=true --id.attrs 'participantType=entity:ecert,entityId=1:ecert,organizationName=studentUnionTecnicoLisboa' --id.secret pw --tls.certfiles ${PWD}/organizations/fabric-ca/org1/tls-cert.pem - echo "## ENROLL studentUnionPresident" - fabric-ca-client enroll -u https://studentUnionPresident:pw@localhost:7054 --enrollment.attrs "participantType,entityId,organizationName" --caname ca-org1 -M ${PWD}/organizations/peerOrganizations/org1.example.com/users/studentUnionPresident@org1.example.com/msp --tls.certfiles ${PWD}/organizations/fabric-ca/org1/tls-cert.pem +echo "## ENROLL studentUnionPresident" +fabric-ca-client enroll -u https://studentUnionPresident:pw@localhost:7054 --enrollment.attrs "participantType,entityId,organizationName" --caname ca-org1 -M ${PWD}/organizations/peerOrganizations/org1.example.com/users/studentUnionPresident@org1.example.com/msp --tls.certfiles ${PWD}/organizations/fabric-ca/org1/tls-cert.pem \end{minted} -This step is essencial for access control within the blockchain. Note how when the client is being registered, we are recording different attributes: \texttt{id.name, id.affiliation, id.attrs, participantType, entityId, organizationName}. Those attributes are going to belong to the X.509 certificate issued to them, which can be later inspected by chaincode. This means that we have a reliable way of identifying participants on the chain and restricting who can access specific functions. For example, we can set up our chaincode in a way that only participants with the attribute \texttt{participantType=university} can call the chaincode function \texttt{createUniversity}. +This step is essential for access control within the blockchain. Note how when the client is being registered, we are recording different attributes: \texttt{id.name, id.affiliation, id.attrs, participantType, entityId, organizationName}. Those attributes will belong to the X.509 certificate issued to them, which can be later inspected by chaincode. This means that we have a reliable way of identifying participants on the chain and restricting who can access specific functions. For example, we can set up our chaincode so that only participants with the attribute participantType=university can call the chaincode function to createUniversity. -After the necessary files are created, the network is bootstrapped, by using Docker. Essencially, the script calls the \texttt{docker-compose \$\{COMPOSE\_FILES\} up -d} command, where the variable COMPOSE\_FILES holds the necessary docker files. +After the necessary files are created, the network is bootstrapped by using Docker. Essentially, the script calls the \texttt{docker-compose \$\{COMPOSE\_FILES\} up -d} command, where the variable COMPOSE\_FILES holds the necessary docker files. The \texttt{docker-compose-test-net.yaml} and \texttt{docker-compose-couch.yaml} are used to create the peers, orderers, CA, and CouchDB images. -The \texttt{docker-compose-test-net.yaml} and \texttt{docker-compose-couch.yaml} are used to create the Peers, Orderers, CA and CouchDB images. +After that, the channel is created, via the \texttt{createChannel} script. It uses the configtxgen tool to generate two-channel artifacts: the genesis block and the first transaction. Those artifacts are configurable through the file configtx.yaml, which is located under config. After that, the channel is created via the command \texttt{peer channel create}, and both organizations join by issuing \texttt{peer channel join}. -After that, the channel is created, via the \texttt{createChannel} script. It uses the configtxgen tool to generate two channel artifacts: the genesis block and the first transaction. Those artifacts are configurable through the file \texttt{configtx.yaml}, located under \texttt{config}. After that, the channel is created via the command \texttt{peer channel create}, and both organizations join, by issuing \texttt{peer channel join}. - -We now have a working blockchain! But for it to be useful, we need to start... +We now have a working blockchain! But for being useful, we need to start... \subsection{Deploying Chaincode} Chaincode has a specific lifecycle, following a well-defined sequence of steps until it is usable: \begin{enumerate} -\item Package chaincode: this step takes as input the source files of the chaincode (those on the chaincode directory), and creates a file (.tar.gz) with them, along with metadata -\item Install chaincode: installs the chaincode package on the Peers -\item Approve chaincode definition: each organization approves the code that was installed -\item Commit chaincode definition to channel: when the lifecycle endorsement policy is achieved (in our case, all entities have to endorse the installed chaincode), an organization commits the new chaincode version. +\item Package chaincode: this step takes to input the source files of the chaincode (those on the chaincode directory) and creates a file (.tar.gz) with them, along with metadata +\item Install chaincode: installs the chaincode package on the peers +\item Approve chaincode definition: each organization approves the code that was installed +\item Commit chaincode definition to channel: when the lifecycle endorsement policy (in our case, all have to endorse the installed chaincode), an organization commits the new chaincode version. \end{enumerate} This process takes place at the \texttt{deployCC.sh} script, inside the \texttt{scripts} folder. In step one, the chaincode is packed: @@ -266,15 +263,15 @@ \subsection{Deploying Chaincode} peer lifecycle chaincode commit [...] \end{minted} -Each step comprises different inputs, that can be analyzed during the lab. After the respective verifications, the script tests invoking a particular smart contracts (via \texttt{chaincodeInvokeInit} and \texttt{chaincodeInvoke}). +Each step comprises different inputs that can be analyzed during the lab. After the respective verifications, the script tests invoking a particular smart contracts (via \texttt{chaincodeInvokeInit} and \texttt{chaincodeInvoke}). \section{Blockchain Client} -We now have a blockchain up and running, which we can interact with using the command line interface (see previous Lab). However, this is not useful for enterprises, organizations, and communities that want to use blockchain technology. To facilitate the interaction with the blockchain, we need to build a Blockchain Client for each organization involved. This Blockchain Client is a web server that interacts with the blockchain via the fabric-sdk. On top of this web server, we deploy a user interface, allowing end-users to interact with the network. Those end users are issued credentials (cryptographic materials discussed before), via the Fabric CA, which are then stored in a wallet. Users utilize their credentials to issue transactions on the blockchain, via their organization's peer node. +We now have a blockchain up and running to interact using the command-line interface (see the previous Lab). However, this is not useful for enterprises, organizations, and communities that want to use blockchain technology. To facilitate the interaction with the blockchain, we need to build a Blockchain Client, for each organization involved. This blockchain client is a web server that interacts with the blockchain via the fabric-sdk. On top of this web server, we deploy a user interface, allowing end-users to interact with the network. Those end-users are issued credentials (cryptographic materials discussed before) via the Fabric CA, stored in a wallet. Users utilize their credentials to issue transactions on the blockchain via their organization's peer node. Let's get started! \subsection{Setting up} -First, we need to add our user identities to their respective wallets. Under the B4S folder, you have a directory called organizations. In a real world deployment, each sub-folder (students-union and university) would belong exclusively to a single stakeholder. Moreover, the crypto materials are generated only for them. On the B4S folder, run the script \texttt{init.sh}. Analyze this script. It essentially installs the necessary dependencies, adds the credentials to the wallet, and issues a transaction on the blockchain, for both organizations. For the students-union: +First, we need to add our user identities to their respective wallets. Under the B4S folder, you have a directory called organizations. In a real-world deployment, each sub-folder (students-union and university) would belong exclusively to a single stakeholder. Moreover, the crypto materials are generated only for them. On the B4S folder, run the script \texttt{init.sh}. Analyze this script. It essentially installs the necessary dependencies, adds the credentials to the wallet, and issues a transaction on the blockchain for both organizations. For the students-union: \begin{minted}[breaklines]{bash} echo '====== ON BEHALF OF A STUDENTS UNION ======' @@ -299,21 +296,19 @@ \subsection{Setting up} \begin{minted}[breaklines]{json} { - "credentials": { - "certificate": "-----BEGIN CERTIFICATE-----[...]\n-----END CERTIFICATE-----\n", - "privateKey": "-----BEGIN PRIVATE KEY-----[...]-----END PRIVATE KEY-----\n" - }, - "mspId": "Org1MSP", - "type": "X.509", - "version": 1 +"credentials": { +"certificate": "-----BEGIN CERTIFICATE-----[...]\n-----END CERTIFICATE-----\n", +"privateKey": "-----BEGIN PRIVATE KEY-----[...]-----END PRIVATE KEY-----\n" +}, +"mspId": "Org1MSP", +"type": "X.509", +"version": 1 } \end{minted} -The script \texttt{showIdentity} issues a transaction against the blockchain requesting for some information. - -Analyze the code of \texttt{showIdentity.js}, under: +The script showIdentity issues a transaction against the blockchain requesting some information. -\texttt{support/Lab06/b4s/organization/university/utils} +Analyze the code of \texttt{showIdentity.js}, under \texttt{support/Lab05/b4s/organization/university}: \begin{minted}[breaklines]{javascript} @@ -322,22 +317,22 @@ \subsection{Setting up} [...] / A wallet stores a collection of identities for use - const wallet = await Wallets.newFileSystemWallet('../identity/org2/wallet'); +const wallet = await Wallets.newFileSystemWallet('../identity/org2/wallet'); - // A gateway defines the peers used to access Fabric networks - const gateway = new Gateway(); +// A gateway defines the peers used to access Fabric networks +const gateway = new Gateway(); \end{minted} -First, we import the Wallet and Gateway classes from the Fabric SDK, in order to interact with the wallet and the blockchain, respectively. We are also using the University class we created. +First, we import the Wallet and Gateway classes from the Fabric SDK, interact with the wallet, and blockchain. We are also using the University class we created. -Now, we need to connect to Fabric, via the user whose credentials were used by the CA. When we generated the connection profile for the user, it was saved on the folder Identity. +Now, we need to connect to Fabric via the user whose credentials were used by the CA. When we generated the connection profile for the user, it was saved on the folder Identity. \begin{minted}[breaklines]{javascript} // Specify userName for network access - const userName = 'universityTecnicoLisboa'; +const userName = 'universityTecnicoLisboa'; - // Load connection profile; will be used to locate a gateway - let connectionProfile = yaml.safeLoad(fs.readFileSync('../gateway/connection-org2.yaml', 'utf8')); +// Load connection profile; will be used to locate a gateway +let connectionProfile = yaml.safeLoad(fs.readFileSync('../gateway/connection-org2.yaml', 'utf8')); \end{minted} @@ -345,27 +340,27 @@ \subsection{Setting up} We now connect to Fabric: \begin{minted}[breaklines]{javascript} - await gateway.connect(connectionProfile, connectionOptions); +await gateway.connect(connectionProfile, connectionOptions); \end{minted} ... to the desired channel... \begin{minted}[breaklines]{javascript} - const network = await gateway.getNetwork('mychannel'); +const network = await gateway.getNetwork('mychannel'); \end{minted} ... obtain the smart contract we need to use... \begin{minted}[breaklines]{javascript} - const contract = await network.getContract('b4s','org.b4s.logistics'); +const contract = await network.getContract('b4s','org.b4s.logistics'); \end{minted} .. and submit the createUniversity transaction! \begin{minted}[breaklines]{javascript} - const universityBytes = await contract.submitTransaction('createUniversity', 'TecnicoLisboa', 'Portugal'); +const universityBytes = await contract.submitTransaction('createUniversity', 'TecnicoLisboa', 'Portugal'); \end{minted} @@ -374,34 +369,30 @@ \subsection{Setting up} Finally, we retrieve the response from Fabric into a human-readable format: \begin{minted}[breaklines]{javascript} - let university = University.fromBuffer(universityBytes); - console.log(university) +let university = University.fromBuffer(universityBytes); +console.log(university) \end{minted} -Now, analyze the code of \texttt{createUniversity.js}, under - -\texttt{support/Lab06/b4s/organization/university/utils}. - -You will notice that it is very similar to \texttt{showIdentity}, only a line changes: +Now, analyze the code of \texttt{createUniversity.js}, under \texttt{support/Lab05/b4s/organization/university}. You will notice that it is very similar to showIdentity, only a line changes: \begin{minted}[breaklines]{javascript} - const response = await contract.submitTransaction('showIdentity'); +const response = await contract.submitTransaction('showIdentity'); \end{minted} That is right, only the transaction type that is called changes. -This procedure applies for all possible transactions defined by a smart contract. We first connect to the network and channel, obtain the smart contract and submit a transaction with our credentials. But for a large organization, it would be very cumbersome to maintain a script for each employee and for each transaction type. That is why we can encapsulate this complexity into a web app. +This procedure applies to all possible transactions defined by a smart contract. We first connect to the network and channel, obtain the smart contract, and submit a transaction with our credentials. But for a large organization, it would be very cumbersome to maintain a script for each employee and for each transaction type. That is why we can encapsulate this complexity into a web app. \subsection{B4S WebApp} -Now, for each organization, there is a web app that takes the generated credentials and interacts with the network. The idea is for the end users to interact with the blockchain in a simple way: the web app does the same than the scripts above. +For each organization, a web app takes the generated credentials and interacts with the network. The idea is for the end-users to interact with the blockchain simply: the web app does the same as the scripts above. -Under \texttt{b4s/organization}, there are two organizations depicted: students-union corresponds to org1, having a wallet that supports two users. University corresponds to org2. Go to the \texttt{university} folder, and then to \texttt{b4s-client}. To start the client, run \texttt{npm run start}. +Under \texttt{b4s/organization}, there are two organizations depicted: students-union corresponds to org1, having a wallet that supports two users. University corresponds to org2. Go to the university folder, and then to b4s-client. To start the client, run npm run start. -The blockchain client is up and running! The ideal situation now would be to have a user interface in which we could interactively issue transactions to the blockchain. For testing purposes, we will issue GET or POST requests to a specific endpoint, to verify that the requests are translated to transactions. Verify the credentials loaded by the b4s-client, by requesting a GET to /logistics/myIdentity in a new terminal: +The blockchain client is up and running! The ideal situation now would be to have a user interface to interactively issue transactions to the blockchain. For testing purposes, we will issue GET or POST requests to a specific endpoint to verify that the requests are translated to transactions. Verify the credentials loaded by the b4s-client by requesting a GET to /logistics/myIdentity: \begin{minted}[breaklines]{bash} curl -X GET http://127.0.0.1:3001/logistics/myIdentity @@ -419,26 +410,26 @@ \subsection{B4S WebApp} Use network channel: mychannel. Use org.b4s chaincode identity: { - participantType: 'university', - auditorId: null, - organizationName: 'universityTecnicoLisboa', - loggerId: null, - adminId: null, - hfEnrollmentID: null, - getID: 'x509::/C=US/ST=North Carolina/O=Hyperledger/OU=client/OU=org2/CN=universityTecnicoLisboa::/C=US/ST=North Carolina/O=Hyperledger/OU=Fabric/CN=fabric-ca-server', - getMSPID: 'Org2MSP' +participantType: 'university', +auditorId: null, +organizationName: 'universityTecnicoLisboa', +loggerId: null, +adminId: null, +hfEnrollmentID: null, +getID: 'x509::/C=US/ST=North Carolina/O=Hyperledger/OU=client/OU=org2/CN=universityTecnicoLisboa::/C=US/ST=North Carolina/O=Hyperledger/OU=Fabric/CN=fabric-ca-server', +getMSPID: 'Org2MSP' } GET /logistics/myIdentity 200 2657.232 ms - 77 \end{minted} -You can check that the transaction was successful by analyzing the peer logs, as well as the logs of the chaincode container. Execute \texttt{docker logs peer0.org1.example.com}, and then +You can check that the transaction was successful by analyzing the peer logs and the logs of the chaincode container. Execute \texttt{docker logs peer0.org1.example.com}, and then \begin{minted}[breaklines]{bash} docker logs dev-peer0.org1.example.com-b4s_1-[SPECIFIC-ID] \end{minted} -replacing SPECIFIC-ID by the ID of your container (you can check this by issuing the command \textttt{docker container ls} and searching for the Container ID of dev-peer0.org1...). +replacing SPECIFIC-ID by the ID of your container (you can check this by issuing docker logs, and searching for dev-peer...). The peer0.org1 container shows: \begin{minted}[breaklines]{bash} 2020-09-10 10:59:48.490 UTC [kvledger] CommitLegacy -> INFO 0cd [mychannel] Committed block [15] with 1 transaction(s) in 34ms (state_validation=0ms block_and_pvtdata_commit=3ms state_commit=29ms) commitHash=[8810dfd0a66bdb305e0932ca3ed80af47472ccdabfc8db02cef23d449fef3f0d] @@ -450,17 +441,17 @@ \subsection{B4S WebApp} The chaincode container shows: \begin{minted}[breaklines]{bash} Show identity called by: x509::/C=US/ST=North Carolina/O=Hyperledger/OU=client/OU=org2/CN=universityTecnicoLisboa::/C=US/ST=North Carolina/O=Hyperledger/OU=Fabric/CN=fabric-ca-server -2020-09-10T10:51:56.134Z info [c-api:lib/handler.js] [mychannel-c0ad8972] Calling chaincode Invoke() succeeded. Sending COMPLETED message back to peer +2020-09-10T10:51:56.134Z info [c-api:lib/handler.js] [mychannel-c0ad8972] Calling chaincode Invoke() succeeded. Sending COMPLETED message back to peer \end{minted} Which corresponds to the console.log done at the chaincode (see showIdentity from the logistics-contract). -This request is semantically equivalent to the script \textttt{showIdentity}. In fact, the client just loads this script, and manages the credentials for you. +This request is semantically equivalent to the script showIdentity. In fact, the client just loads this script and manages the credentials for you. -Now, we want to perform more complex transactions. For instance, we can create a university by requesting a POST to /logistics/createUniversity, containing the necessary arguments. +Now, we want to perform more complex transactions. For instance, we can create a university by requesting a POST to /logistics/createUniversity, containing the necessary arguments. \begin{minted}[breaklines]{bash} curl -X POST -F 'universityId=123' -F 'regionId=Portugal' http://127.0.0.1:3001/logistics/createUniversity @@ -472,11 +463,11 @@ \subsection{B4S WebApp} =============== University { - class: 'org.b4s.university', - key: '"123":"Portugal"', - currentState: null, - universityId: '123', - regionId: 'Portugal' +class: 'org.b4s.university', +key: '"123":"Portugal"', +currentState: null, +universityId: '123', +regionId: 'Portugal' } Transaction complete. POST /logistics/createUniversity 200 2605.641 ms - 122 @@ -490,11 +481,11 @@ \subsection{B4S WebApp} -That's it! Notice that there is also a b4s-client for students-union, under their directory. You can repeat this experiment, but notice that the students client only contains the myIdentity endpoint. We are now leveraging a user interface from the University institution to render the results from the getAllStudents endpoint, which retrieves a full list of enrolled students. Go to /organizations/university/user-interface. To install dependencies, \textttt{run npm install}. After that, bootstrap the user interface by running \textttt{npm run serve}. (Note, if this step does not work, install the Vue Client\footnote{https://vuejs.org/v2/guide/installation.html}). It is out of the scope of this course to learn about Vue.js, but you can check the code at University-frontend.vue, under src/Views to inspect how it works. Essentially, it makes a request to http://localhost:3001/users/getAllStudents, which has credentials loaded, and parses the request. +That's it! Notice that there is also a b4s-client for students-union under their directory. You can repeat this experiment but notice that the students' client only contains the myIdentity endpoint. We are now leveraging a user interface from the University institution to render the results from the getAllStudents endpoint, which retrieves a full list of enrolled students. Go to /organizations/university/user-interface. To install dependencies, run npm install. After that, bootstrap the user interface by running npm run serve. (Note, if this step does not work, install the Vue Client\footnote{https://vuejs.org/v2/guide/installation.html}). It is out of this course's scope to learn about Vue.js, but you can check the code at University-frontend.vue, under src/Views to inspect how it works. Essentially, it makes a request to http://localhost:3001/users/getAllStudents, which has credentials loaded and parses the request. -Open your browser at localhost:8080, and try out to retrieve the students list. +Open your browser at localhost:8080, and try out to retrieve the students' list. -That's it! These are the basics about Hyperledger Fabric. +That's it! These are the basics of Hyperledger Fabric. @@ -502,7 +493,7 @@ \subsection{B4S WebApp} \section{Clean up} On the B4S folder, run: \texttt{./network-clean}. This calls \texttt{network.sh down}, that tears down the whole infrastructure, as removes all stopped containers. -For each organization, you may run the script remove\_wallet.sh, under the \texttt{utils} folder. This deletes the created credentials. +For each organization, you may run the script remove\_wallet.sh, under the \texttt{utils} folder. This deletes the created credentials. @@ -512,7 +503,7 @@ \section{Clean up} %\subsection{CA} %\subsection{Channels} -%Hyperledger Fabric allows an organization to simultaneously participate in multiple, separate blockchain networks via channels. By joining multiple channels, an organization can participate in a so-called network of networks. Channels provide an efficient sharing of infrastructure while maintaining data and communications privacy. They are independent enough to help organizations separate their work traffic with different counterparties, but integrated enough to allow them to coordinate independent activities when necessary. +%Hyperledger Fabric allows an organization to simultaneously participate in multiple, separate blockchain networks via channels. By joining multiple channels, an organization can participate in a so-called network of networks. Channels provide an efficient sharing of infrastructure while maintaining data and communications privacy. They are independent enough to help organizations separate their work traffic with different counterparties but integrated enough to allow them to coordinate independent activities when necessary. %\section{Off-Chain Components} %\subsection{Blockchain Client} @@ -540,12 +531,11 @@ \section{Clean up} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % YOUR TEXT ENDS HERE -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Exercises} -%How to add the students grades in order to not only provide reputation on answering the QUCs, but also on their grades (like Qualichain). \subsubsection*{The \texttt{network.sh} script can be reproduced step by step\footnote{https://medium.com/@kctheservant/add-a-peer-to-an-organization-in-test-network-hyperledger-fabric-v2-2-4a08cb901c98}\footnote{https://hyperledger-fabric.readthedocs.io/en/release-2.2/tutorial/commercial\_paper.html}. Reproduce the process described on the B4S network.} @@ -553,7 +543,7 @@ \subsubsection*{Suppose that we want to add a new peer node to the network. Refe \subsubsection*{In the deployCC.sh script, what is the difference between chaincodeInvoke and chaincodeInvokeInit?} -\subsubsection*{What are some dangers of using the web app as an centralized party to access the blockhain?} +\subsubsection*{What are some dangers of using the web app as a centralized party to access the blockhain?} \subsubsection*{What are the differences of this project to a production environment?} diff --git a/source/Lab06/Instructors Guide/assignment.tex b/source/Lab06/Instructors Guide/assignment.tex index 01eaaa9..3ba088a 100644 --- a/source/Lab06/Instructors Guide/assignment.tex +++ b/source/Lab06/Instructors Guide/assignment.tex @@ -2,7 +2,7 @@ % AUTHOR: Rafael Belchior % Thanks to Prof. RUI SANTOS CRUZ for providing the template % -\usepackage{helvet} +\usepackage{helvet} \renewcommand{\familydefault}{\sfdefault} \usepackage{a4wide} \usepackage{ucs} @@ -19,8 +19,8 @@ \usepackage{hyperref} \usepackage{cite} \usepackage[absolute]{textpos} -\usepackage{tabularx} -\usepackage{tabulary} +\usepackage{tabularx} +\usepackage{tabulary} \usepackage{fancyhdr} \usepackage[table]{xcolor} \pagestyle{fancy} @@ -70,16 +70,16 @@ \lhead{} {\renewcommand{\arraystretch}{1.1} \fancyhead[C]{\begin{tabularx}{1.0\textwidth}{|l|X|l|l|} -\hline +\hline % In the following line change Course Name: PPIII, PPB \textbf{EB 20/21} & \textbf{Enterprise Blockchain Technologies} & \textbf{Number:} & 6\\ \hline % In the following line insert your Name and IST ID -\multicolumn{2}{|l|}{Module II - Hyperledger Fabric} & \textbf{Issue Date:} & - \\ +\multicolumn{2}{|l|}{Module II - Hyperledger Fabric} & \textbf{Issue Date:} & - \\ \hline % In the following line insert the Activity CODE and Title (abridged) %\textbf{WP n.} (99) & (Subject) & \textbf{Group:} & (99) \\ -\multicolumn{2}{|l|}{Hyperledger Fabric: Full-Stack dAp} & \textbf{Due Date:} & - \\ +\multicolumn{2}{|l|}{Hyperledger Fabric: Full-Stack dApp} & \textbf{Due Date:} & - \\ \hline \end{tabularx}} \rhead{} @@ -94,34 +94,32 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%,sdist2017 \section*{Instructors Guide} -The majority of this laboratory was inspired on the implementation of JusticeChain\cite{belchior2019_audits}. Part of this laboratory was inspired in sources elaborated by KC Tam\footnote{https://medium.com/@kctheservant}. +The majority of this laboratory was inspired by the implementation of JusticeChain\cite{belchior2019_audits}. Part of this laboratory was inspired in sources elaborated by KC Tam\footnote{https://medium.com/@kctheservant}. \section{Exercises} -%How to add the students grades in order to not only provide reputation on answering the QUCs, but also on their grades (like Qualichain). - \subsubsection*{The \texttt{network.sh} script can be reproduced step by step\footnote{https://medium.com/@kctheservant/add-a-peer-to-an-organization-in-test-network-hyperledger-fabric-v2-2-4a08cb901c98}\footnote{https://hyperledger-fabric.readthedocs.io/en/release-2.2/tutorial/commercial_paper.html}. Reproduce the process described on the B4S network.} \subsubsection*{Suppose that we want to add a new peer node to the network. Referring to Figure 3, which steps are to be taken?} -A: +A: Step 2: generate crypto material for the peer -Step 4: create a docker compose file containing this peer +Step 4: create a docker-compose file containing this peer Step 5c: join this new peer to the existing channel Step 6b: install chaincode to that peer \subsubsection*{In the deployCC.sh script, what is the difference between chaincodeInvoke and chaincodeInvokeInit?} -A: chaincodeInvokeInit is a convention that was widely used on Fabric 1.4 (as every instance of the blockchain needed a transaction to initialize the ledger). In Fabric 2.x, there is no need of this step. In practise, chaincodeInvoke and chaincodeInvokeInit are used to test if there are no error with the deployment, as they call different methods on the org.b4s.quc smart contract. However, the peer command called is exactly the same (peer chaincode invoke). +A: chaincodeInvokeInit is a widely used convention on Fabric 1.4 (as every instance of the blockchain needed a transaction to initialize the ledger). In Fabric 2.x, there is no need for this step. In practice, chaincodeInvoke and chaincodeInvokeInit are used to test if there is no error with the deployment. They call different methods on the org.b4s.quc smart contract. However, the peer command called is precisely the same (peer chaincode invoke). -\subsubsection*{What are some dangers of using the web app as an centralized party to access the blockhain?} -A: The transactions are issued accordingly to the arguments passed by the web app. If the web app is not open-source, we may not know what is being passed to the blockchain. +\subsubsection*{What are some dangers of using the web app as a centralized party to access the blockchain?} +A: The transactions are issued accordingly to the arguments passed by the web app. If the web app is not open-source, we may not know what is being given to the blockchain. \subsubsection*{What are the differences of this project to a production environment?} -A: Some of them: A backoffice may exist, where organizations could add/reomve channels, participants, and peer nodes, in a distributed setting. The frontend could dinamically load user credentials: users could use a card with their credentials to sign credentials, without the system having to hold them. The cripto material is generated by different organizations and not shared. There might be different web applications and interfaces, depending on the organizations. There is chaincode access control (for example, only specific users may be able to access certain functions). +A: Some of them: A back office may exist, where organizations could add/remove channels, participants, and peer nodes, in a distributed setting. The frontend could dynamically load user credentials: users could use a card with their credentials to sign credentials without the system having to hold them. The crypto material is generated by different organizations and not shared. There might be various web applications and interfaces, depending on the organizations. There is chaincode access control (for example, only specific users may access certain functions). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \bibliographystyle{IEEEtran} \bibliography{lab.bib}