alamin/websurfx
alamin/websurfx
0
0
Fork 0
mirror of https://github.com/neon-mmd/websurfx.git synced 2024-10-18 06:22:53 -04:00
Code Issues Releases Activity

Merge pull request #1 from alamin655/rolling

Browse Source 
Improve documentation clarity and completeness
This commit is contained in:
neon_arch 2023-04-28 15:58:21 +00:00 committed by GitHub
commit a412a756a8
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 60 additions and 60 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
CONTRIBUTING.org
View File

@ -1,24 +1,26 @@
* Things to consider before contributing.
* Things to Consider Before Contributing
** Knowledge Required
- Rust basics.
- Actix-web crate basics.
- Tokio crate and async/await.
- Reqwest crate basics.
- Serde and serde_json crate basics.
- fake_useragent crate basics.
- pyo3/hlua/rlua crates basics.
- Rust basics
- Actix-web crate basics
- Tokio crate and async/await
- Reqwest crate basics
- Serde and serde_json crate basics
- fake_useragent crate basics
- pyo3/hlua/rlua crates basics
** Guidelines
- Please be patient.
- Treat everyone with respect --- ("give respect and take respect").
- Treat everyone with respect -- "give respect and take respect."
- Document your code properly with rust coding conventions in mind.
- Document your code properly with Rust coding conventions in mind.
- Provide a brief description of what changes you made in the pull request.
- Provide a brief description of the changes you made in the pull request.
- Provide an apropriate header to pull request.
- Provide an appropriate header for the pull request.
*NOTE:* The rolling branch is where all the contributions should go. In simple terms, it is the working branch for this project.
*NOTE:* The rolling branch is where all contributions should go. In other words, it is the working branch for this project.
We appreciate any contributions and suggestions to help improve the Websurfx project. Please keep in mind the above requirements and guidelines before submitting a pull request.

36
README.org
View File

@ -1,6 +1,6 @@
* Websurfx
A lightening fast, privacy respecting, secure [[https://en.wikipedia.org/wiki/Metasearch_engine][meta search engine]]. (pronounced as websurface or web-surface /wɛbˈsɜːrfəs/.)
Websurfx is a lightning-fast, privacy-respecting, secure [[https://en.wikipedia.org/wiki/Metasearch_engine][meta search engine]] (pronounced as websurface or web-surface /wɛbˈsɜːrfəs/.) written in Rust. It provides a fast and secure search experience while respecting user privacy.
* Preview
@ -18,66 +18,66 @@ A lightening fast, privacy respecting, secure [[https://en.wikipedia.org/wiki/Me
* Installation and Testing
To start installing, testing and playing around with search engine. Clone the repo and then *cargo run* as shown below:
To get started with Websurfx, clone the repository and run it using the following commands:
#+begin_src shell
git clone https://gitlab.com/NEON-MMD/websurfx.git
git clone https://github.com/neon-mmd/websurfx.git
cd websurfx
cargo build
./target/debug/websurfx --port 8080
#+end_src
If you want to be on the rolling/edge branch then do the following steps:
If you want to use the rolling/edge branch, run the following commands instead:
#+begin_src shell
git clone https://gitlab.com/NEON-MMD/websurfx.git
git clone https://github.com/neon-mmd/websurfx.git
cd websurfx
git checkout rolling
cargo build
./target/debug/websurfx --port 8080
#+end_src
and then open your browser of choice and visit [[http://127.0.0.1:8080]] and then you can start playing with it right away.
Once you have started the server, open your preferred web browser and navigate to http://127.0.0.1:8080 to start using Websurfx.
*Important Note:* Do not over refresh your page or put too many requests or start hosting on production server as there are no behaviours right now in the code to prevent ip blocking (it is still work under process) and you can potential can get your ip blocked/banned and also the project is still under testing phase and far from being complete and ready to be hosted on production machines.
*Important Note:* Please be aware that the project is still in the testing phase and is not ready for production use. Do not refresh the page excessively or make too many requests, as this could result in IP blocking.
* More Contributers Wanted
I am looking for more willing contributors for this project so that it helps grow this project. For more information on what you can contribute this project check out the [[file:goals.org][goals.org]] and also check out the [[file:CONTRIBUTING.org][CONTRIBUTING.org]] for guidelines and rules for making the contribution.
We are looking for more willing contributors to help grow this project. For more information on how you can contribute, check out the [[file:goals.org][goals.org]] file and the [[file:CONTRIBUTING.org][CONTRIBUTING.org]] file for guidelines and rules for making contributions.
* FAQ (Frequently Asked Questions)
** Why Websurfx?
The main goal of the project is to provide a fast, secure and privacy focused [[https://en.wikipedia.org/wiki/Metasearch_engine][meta search engine]]. Though there are many meta search engine out there but they don't provide gaurantee security of the their search engine which is essential because sometimes privacy is related to security like for example some memory vulnerabilities can leak private or sensitive information which is never good so the project being written in rust gaurantees memory safety and thus eliminating such problems and also many meta search engines lack many features like advanced image search *(which is required by may graphics designers, content creators, etc), proper nsfw blocking (many links are still visible even on strict safe search), etc which *websurfx* aims to provide.
The main goal of the Websurfx project is to provide a fast, secure, and privacy-focused [[https://en.wikipedia.org/wiki/Metasearch_engine][meta search engine]]. While there are many meta search engines available, they do not always guarantee the security of their search engine, which is essential for ensuring privacy. For example, memory vulnerabilities can leak private or sensitive information, which is never good. Websurfx is written in Rust, which guarantees memory safety and eliminates such problems. Many meta search engines also lack key features such as advanced image search, which is required by many graphic designers, content creators, and others. Websurfx aims to provide these features and others, such as proper NSFW blocking, to improve the user experience.
** Why AGPLv3?
The reason why I am releasing my meta search engine under *AGPLv3* is because my main motive of this meta search is to provide privacy, speed and security to the user and *GPL licensing or closed sourcing or any other licensing* makes it loose for the user/server admin to not release the source code which is shady because you never know what he is putting in the source code lets say he is adding telemetry, spyware stuff, logging and what not which gets shady and is against privacy.
Websurfx is released under the *AGPLv3* license to ensure that the source code remains open and transparent. This helps to prevent the inclusion of spyware, telemetry, or other malicious code in the project. *AGPLv3* is a strong copyleft license that ensures the source code of the software remains open and available to everyone, including any modifications or improvements made to the code.
** Why Rust?
Rust provides memory safe code which eliminates issues relating memory based vulnerabilites which makes the project more secure and less vulnerable to memory based bugs and issues and also rust being a faster programming language than c++ provides a much faster, smooth and secure meta search engine :).
Rust was chosen as the programming language for Websurfx due to its memory safety features, which can help prevent vulnerabilities and make the codebase more secure. Rust is also faster than C++, which helps to make Websurfx fast and responsive. In addition, Rust's ownership and borrowing system allows for safe concurrency and thread safety in the codebase.
* Contributing
Contributions are welcome. It does not matter who you are you can still contribute to the project in your way :).
Contributions are welcome from anyone. It doesn't matter who you are; you can still contribute to the project in your way.
** Not a developer but still want to contribute
** Not a developer but still want to contribute?
Here is [[https://youtu.be/FccdqCucVSI][video]] by Mr. Nick on how to contribute and credit to him as well
Check out this [[https://youtu.be/FccdqCucVSI][video]] by Mr. Nick on how to contribute.
** Developer
If you are developer, have a look at the [[file:CONTRIBUTING.org][CONTRIBUTING.org]] document for more information.
If you are a developer, have a look at the [[file:CONTRIBUTING.org][CONTRIBUTING.org]] document for more information.
* License
The project is available under the [[file:LICENSE][GPLv3]] license.
Websurfx is available under the [[file:LICENSE][AGPLv3]] license.
* Credits
Thanks to this wonderful people for *contributing* and *supporting* this project:
We would like to thank the following people for their contributions and support:
- [[https://gitlab.com/XFFXFF][zhou fan]]
- [[https://github.com/XFFXFF][zhou fan]]
- [[https://gitlab.com/johannesrexx][johannes rex]]

54
goals.org
View File

@ -1,57 +1,55 @@
* TODO Goals for v0.1.0
- [ ] Add unit tests, integration test, doc tests in the project source code.
- [ ] Add unit tests, integration test, and doc tests to the project source code.
- [ ] Add code to remove nsfw content from search results using a blocklist.
- [ ] Add code to remove NSFW content from search results using a blocklist.
- [ ] Add code to disallow user to search sensitive content (similar functionality to swisscows search engine) if strict safe search is turned on.
- [ ] Add code to disallow user from searching for sensitive content (similar functionality to swisscows search engine) if strict safe search is turned on.
- [ ] Add better error handling code to handle scraping error, reqwest error, etc.
- [ ] Improve error handling to handle scraping errors, reqwest errors, and other issues.
- [ ] Add ability to change colorschemes with the theme style of the page.
- [ ] Add the ability for users to change the colorscheme of the page.
=For example:=
If there is simple theme is there then there is option of 9 different colorschemes to choose from such as catppuccin-mocha, solarized dark, nord, etc.
If a simple theme is used, then there should be the option of 9 different colorschemes to choose from, such as Catppuccino-Mocha, Solarized Dark, Nord, etc.
- [ ] Add random delays and behaviours to emulate human behaviour if needed to evade ip blocking.
- [ ] Implement random delays and behaviors to emulate human behavior to evade IP blocking.
- [ ] Add python/lua config to the search engine and also ensuring and giving more control to the user (server maintainer/administrator).
- [ ] Add Python/Lua configuration options to give more control to the user (server maintainer/administrator).
- [ ] Add settings page to configure search engine on the fly by the user and save his preferences using cookies.
- [ ] Create a settings page to allow users to configure the search engine and save their preferences using cookies.
- [ ] Add search engine logo to index page and to the navbar (on the right hand side).
- [ ] Add a search engine logo to the index page and the navbar.
- [X] Write rust.yml for automating testing of the project.
- [X] Write a rust.yml file for automated testing of the project.
- [X] Add better documentation to source code.
- [X] Improve documentation for the project source code.
- [X] Add FAQ section in readme.org for answering commonly asked questions like why websurfx?, why rust?, why GPLv3?, etc.
- [X] Add a FAQ section to the README.org file to answer common questions about the project.
- [X] Add logging to websurfx project.
- [X] Add logging to the websurfx project.
- [X] Add code to generate random user agent to protect user's privacy.
- [X] Generate random user agents to protect user privacy.
- [X] Add duckduckgo engine as an upstream.
- [X] Add DuckDuckGo and at least one Searx engine instance as upstream providers.
- [X] Add atleast one searx engine instance as an upstream engine.
- [X] Add pagination support for search results.
- [X] Add pagination support.
- [X] Create basic Handlebars pages and theme with Catppuccino colorscheme.
- [X] Add basic handlebars pages and theme with catppuccin colorscheme.
* Goals for future development
* Goals for the future
- Replace Handlebars with the faster templating engine /Tera/.
- Move from handlebars to faster templating engine /Tera/.
- Add more upstream search engines.
- Adding more upstream search engines.
- Implement dorking support (like Google).
- Add dorking support (like google).
- Add advanced search functionality and a dropdown menu for it.
- Add advanced search functionality and dropdown menu for it.
- Add more categories to the search engine's search page, such as images, files, news, and maps.
- Add more categories to search engine's search page like images, files, all, news, maps, etc.
- Add advanced image functionality for content creators, video editors, etc.
- Add advanced image functionality to images category (which will be a great aid for content creators, video editors, etc.).
- Add GPT integration (taking insipiration from langchain module of python or incorporating it using pyo3) and give user the choice to add api key (giving user the choice is important as there are many people who are against ai and so this will ensure that those people also get satisfied like if they don't gpt so they will not their api key and so it will be disabled).
- Add GPT integration, taking inspiration from the Langchain module of Python or incorporating it using pyo3, and give users the choice to add an API key. Giving users the choice is important, as some people may be against AI, and this will ensure that those users who don't want to use GPT can disable it by not adding an API key.