-
Vertical growth of processors has reached a limit, but we can still scale them horizontally by attaching new devices, or cloud services to quench our thirst for computing power. But this too is expensive, and can fast burn a hole in an individual’s pocket
-
Here comes the concept of volunteer computing- a lot of devices in today’s world are idle, the computers in our computer labs, for example, and these idle devices can be used for our computing needs, that’s it
-
Dis-Trie-Buted revolves around connecting these idle computing resource providers, to those in need, and even provide a layer of incentivization to benefit the provider, but not as expensive as cloud services, so that the solution is still viable.
- Create New Task
- Set the Incentive
- Add Worker File
- Select The Task
- Leave The Task Running
- Claim Your Incentive
- The main file being executed on the master node. This file handles all the routes, there are only two ->
/and/:namespace. - For the
/route -> Parses through the current problems available dynamically and fetchs their data fromconfig.jsonfile to display on main page, and renderesmain.ejswith that data - For the
/:namespace-> Handles the communication, by creating namespace for each and every problem, to differentiate between communication lines, and also passes config to the worker files, and rendersworker.ejswhich has links to the other files ->socket.jsandworker.js. - Messages exchanged are ->
connection>initialize>ready>range>processingDone - Note that this index.js is a common file for all sorts of problem and also it is executed ONLY on the master node, and not on the slave nodes
- This is the common file executed for all problems on the slave node
- This sets up the communication channels for both, master-slave as well as web worker communication, and all of these communications are standardized for all problems, so they are common
- Generates a team of web workers, as per the number of cores on the machine, and assignes each one a set of index to work with, so effectively only communicate once with each worker, for a piece of task.
- Handles status of workers as well, and termination as well explicitly, as workers don't have any internal functions for that.
- Only pass parameters to each worker, and they generate the array of tasks to perform from that, saving communication costs
- This is the file that will be different for each problem, and shall be provided by the client who needs to use our platform
- This will contain the mechanism as to how to generate the input, from the passed parameters, whether need to read from external file or generate implicitly, all done here
- The actual algorithm and what is returns, and when to message the main thread about it
- Note that once main thread receives a single message from the worker thread it is closed down, so need to send all messages at once.
- The upside is faster communication, the downside being if the node goes offline all partial work done is lost as well
- Provided by the client for each problem
- Contains various information as -
start,end,step,rewardswhich helps to setup the platform
connection- Executed when the slave node connects for the first time to master node, usingio.connect(url)initialize- Once the master node receives connection request, it sends initialize request along withurlwhich contains the algorithm or problem to be executedready- The slave node receives the algorithm from the url and creates worker threads from it and initializes other parameters, and sends a ready request to the master noderange- Finally master node sends the range of data, or the data to work on to the slave node using the range messageprocessingDone- When the whole range is calculated by the slave node, sends a message ofprocessingDoneto master node.
- Neel Shah - nilshah98
- Shivam Pawase - shivam1708
- Chirag Shetty - nurdtechie98