A chrome extension that identifies fake news. Incomplete.
Go to the extensions page of your browser, in Chrome chrome://extensions
, toggle developer mode to on, then Load unpacked
and choose the folder of this project.
The manifest.json
is the configuration file for chrome extensions. Features may not work properly if that isn't set up properly. It is where the extension's permissions, popups, and details such as name and description are configured. We're roughly following the documentation for Chrome Manifest V3.
icons
- The directory where the extensions icons are, referenced bymanifest.json
popup
- The directory for the code of the popup page which appears when the extension is opened or clicked in the extensions bar. This is simply a webpage but with a specialchrome
object predefined.settings
- The directory for the settings page of the extension. This is simply a webpage but with a specialchrome
object predefined.foreground.js
- This script is ALWAYS injected into webpages that are opened and matcheshost_permissions
inmanifest.json
inject.js
- This script is not actually injected into webpages but can be injected usingchrome.scripting.executeScript
method.service-worker.js
andservice-worker-utils.js
- This script runs in the background, useful for caching things manually, but otherwise can be left alone.
The process is very similar to how ElectronJS applications are made (desktop apps written through web technologies). As a comparison, the electron frontend here is the "popup" which is in an isolated environment (sandboxed and has lesser features), meanwhile to access the actual webpage of the client, we have "content scripts" (foreground.js
and inject.js
) and we just pass the data along from and to using an API.
- The client clicks the extension and the extension popup appears.
- The client clicks the button inside the popup.
- The popup injects code to the webpage the client is on using Chrome's Scripting API.
- The injected scripts scrapes the necessary content and sends it back to the popup using Chrome's Messaging API.
- The popup receiving the content, can now process and display it.
Web scraping is done in a smart manner. The client's device does all the work for web scraping. That's the main purpose of inject.js
which injects a script that identifies content and sends the result back to the popup through the Chrome's Messaging API.
Now how do we configure the web scraping script? A lot of big sites would dynamically render their pages with autogenerated classnames which are randomized. That in itself is already powerful in deterring web-scraping. However, there is always a consistent structure present among these dynamically rendered pages.
Take for example, in React
, we have a Post
component that requires an author's image, author, and a description. And in that particular application, we have a lot of posts being rendered in a single page. The thing is, even if we autogenerate the classnames, the structure of the posts would all be similar or even the same in terms of markup. This is why the web scraping is done by doing a smart pattern matching that uses a sequence of CSS queries to describe the element we need. It finds its cousin's and automatically is able to find its parent.
Here is the code for clarification (refer to inject.js
):
// IMPORTANT: this query sequence works specifically when you open a facebook post
const matches = matchElementsOfQuerySequence([
'a svg image', // profile picture - not text
'h2 span a', // profile name
'span a span', // post time
'div span', // unknown - skippable
'div span', // post privacy
'div span', // post content
'div span', // unknown - skippable
], document, true); // true means this array will only have at most 1 element
As you can see we just specified a sequence of elements using a CSS query of their tags, we do not need to know their exact location, id or classes, we just need to roughly know how to get to them. The algorithm ensures that it gets the elements in sequence and conservatively widens its scope to find its cousins or sibling elements matching the criteria. The output of matches
is a 2d array of elements where the number of columns is specified by how many queries we provided (7 in the example), so it returns an array like so:
[ [ 7 x Element ] x N ] // where N is the number of matches or [0 to 1] if firstMatchOnly parameter is true
This means to support different pages we can use a different query sequence instead like so:
let facebookPostInside = [ 'a svg image', 'h2 span a', 'span a span', 'div span', 'div span', 'div span', 'div span' ];
let facebookPostOutside = [ /* insert query sequence here can be found through inspect element */ ];
let twitterPostInside = [ /* insert query sequence here can be found through inspect element */ ]
// and since this is an injected script we have access to document.location
if (/* check through document.location if inside a facebook post possibly through regex */) {
let matches = matchElementsOfQuerySequence(facebookPostInside, document);
// do something with the matches ...
}
That's pretty much it.
This repository contains a minimal Chrome/Chromium extension that uses the newest version of the manifest (v3).
You can use it as a basis to develop an extension. It can also give you more insights about how to turn a v2 extension to v3.
In other words, this is a working, installable v3 extension example meant for you to jumpstart the development of your own extension.
- Fork this repo, then clone your forked repo locally. If you don't have a github account, you can simply download a zip of the repo and unzip it on your computer.
- Open the extensions page in your browser:
chrome://extensions
. This link works on any chromium-based browser. - If you did not do it already, toggle the "developer mode". This is usually a toggle button at the top right of the extensions page.
- Click the button load unpacked extension.
- In the window that pops up, select the folder that contains this minimal extension, then click ok.
- Done! A new extension called Chrome Addon v3 Starter should have appeared in the list.
Does this work only on Chrome or on other web browsers as well?
At the moment, this works on every chromium-based web browser that supports v3 extensions. Therefore, you should be able to install this extension on any of the following browsers (as long as they are up-to-date):
- Free and open-source browsers:
- Chromium
- Brave
- Proprietary browsers:
- Chrome
- Edge
- Vivaldi
- Opera
So it doesn't work on Firefox or Safari?
No, Firefox uses a different extension format. That being said, it is usually not too hard to port extensions from Chrome to Firefox. Read their porting documentation for more information.
Safari uses yet another extension format and porting is usually harder. You can find more information here.
Does this work on Chrome for Android/iOS?
Chrome for mobile doesn't currently support extensions.
I don't need a popup tool for my extension! Can I remove it?
Yes, simply delete the popup
folder and remove the default_popup
property from the manifest.
I changed some code in the extension, but my changes aren't taken into account!
For most of the changes you make, you will need to reload your extension for the changes to be applied. To do that, go to the chrome://extensions page and click the reload button of your extension. Note that most of the changes you will make to the settings page or the popup don't require reloading the extension.
Can I follow a tutorial about a v2 extension with this?
Most of what you will find in those tutorials still holds with v3.
However, a few things (notably best practices) have changed. You should read the official migration page (v2 to v3) before following such a tutorial.
When I make an erroneous change in my service worker, the extension doesn't load! How can I debug a service worker?
Using the debugger if your service worker is not loaded is not possible.
However, if you want to test some piece of code before putting it in your service worker, you can:
- load your extension with a working version of the service worker.
- click on "service worker" on the page chrome://extensions. This will open the console attached to the service worker.
- paste your code in the console and see if any error is logged.
Note that in this console, you have access to anything your service worker has access to, including its variables, functions and chrome APIs.
How do I uninstall this extension from my browser?
- Go to the extensions page: chrome://extensions. There should be a card with the name of this extension as title. If you don't see such a card, it means the extension is not installed.
- Simply click the Delete button at the bottom of the card. Click ok if a popup asks you for confirmation. The extension is now uninstalled.
I want to push my changes to my own repo, how do I do this?
-
If you forked this repo and cloned your own fork locally, git will push to your fork on your account automatically (i.e. use the command
git push
orgit push origin <your-branch>
). -
If you downloaded a zip or simply cloned this repo locally, do the following:
- Create a github account if you don't already have one and install git on your machine.
- Create a new (empty) repo on your github and copy its url.
- Open a terminal in the folder where the extension is cloned.
- Run the command
git init
, thengit commit -am "Initial commit"
- Run the command
git remote add origin <url-of-your-repo>
- Run
git push -u origin master
. The extension code is now on your repo, on brnach master. - If you want, you can make the master branch the default one and delete the main branch in the settings of your repo.