0
0
mirror of https://github.com/neon-mmd/websurfx.git synced 2024-12-22 20:38:22 -05:00
Commit Graph

294 Commits

Author SHA1 Message Date
neon_arch
fda6c3a9be feat: rename public_paths to paths (#163) 2023-08-22 19:18:43 +03:00
neon_arch
4f28711218 feat: add code to filter aggregated search results using lists (#163) 2023-08-22 19:16:37 +03:00
Milim
87e230d731 fix debug level not being set properly 2023-08-19 09:09:09 +02:00
Milim
21403d0a48 fix dumb rustfmt version and bump dependency versions 2023-08-18 22:04:34 +02:00
Milim
5aca5c0d0d Improve aggregation
Adds the EngineHandler struct
Removes vulnerability where an attacker could send requests cookies with fake engine names and crash the server.
Merged RawSearchResult and SearchResult, as they were functionally identical.
2023-08-18 10:43:53 +02:00
Milim
15dfda6ea9 Improve Aggregation function & config parser
Refactor aggregation function
Rename visiting_url to url, as they are always the same (see upstream engine scalping).
Refactor parsing function to be more readable.
2023-08-17 22:48:20 +02:00
Milim
af3b1cb308 Use non-blocking sleep function
std:🧵:sleep blocks the entire thread and thus also forces all other current requests to be halted.
2023-08-17 21:45:48 +02:00
neon_arch
c4935f202a feat: add images, error_box & new message when no results are provided (#185) 2023-08-10 04:32:47 +03:00
neon_arch
272fdefff5
Merge branch 'rolling' into improve-results-caching 2023-08-06 20:53:14 +03:00
neon_arch
a596c07547 🔧 chore: add code to cache the previous & next pages (#160) 2023-08-06 20:31:30 +03:00
neon_arch
ca13fb7ad4 Merge branch 'improve-logging-based-on-levels-and-opts' of github.com:neon-mmd/websurfx into improve-logging-based-on-levels-and-opts 2023-08-05 19:20:58 +03:00
neon_arch
653d08c801 🔧 chore: fix link in the logs & add logs for engine error (#170) 2023-08-05 19:16:27 +03:00
neon_arch
0d8c12e26b
Merge branch 'rolling' into improve-logging-based-on-levels-and-opts 2023-08-04 12:55:05 +03:00
neon_arch
9d3a8e065a
Merge branch 'rolling' into patch-csrf-security-with-cors 2023-08-04 12:35:26 +03:00
neon_arch
8b0f2f50f0 🔧 chore: improve logging based on levels & config options (#173) 2023-08-03 23:09:08 +03:00
neon_arch
bef8956da6 🛠️ fix: add code to prevent csrf attacks using cors (#172) 2023-08-03 17:44:13 +03:00
neon_arch
e02fafd69c feat: fix doc tests 2023-08-03 01:03:56 +03:00
neon_arch
4b4dc28cd2 feat: add control check to avoid reinitializing logging & fix tests 2023-08-03 00:12:09 +03:00
neon_arch
11bcf9c98a feat: remove space from the end of the line 2023-08-02 20:07:29 +03:00
neon_arch
d8bd2feb6e feat: add new config option to manage threads and improve logging 2023-08-02 20:05:39 +03:00
neon_arch
a8b6a9bce2
Merge branch 'rolling' into patch-searx-results-on-first-page 2023-07-31 13:57:34 +03:00
neon_arch
a8791deb13 🛠️ fix: add & improve code to handle page handling in searx & duckduckgo 2023-07-30 20:14:40 +03:00
neon_arch
9cb582a10d feat: improve code documentation 2023-07-30 17:08:47 +03:00
neon_arch
25d6af0251 🧹 chore: reformat code to make rustfmt happy 2023-07-30 16:58:19 +03:00
neon_arch
2d47e8d730 feat: provide the functionality to use the new config option 2023-07-30 10:53:48 +03:00
neon_arch
6d3396b6f9 feat: provide functionality to parse the new config option 2023-07-30 10:52:03 +03:00
neon_arch
a28d559426 feat: add documentation for the outputs.iter() code block 2023-07-17 17:03:50 +03:00
neon_arch
987e667c3d feat: rename results_json to results 2023-07-17 13:17:24 +03:00
neon_arch
7e3b59e8c0 feat: remove the empty_result_set option and helper functions 2023-07-17 13:03:20 +03:00
neon_arch
c5fca32553 feat: fix spelling mistake in the comment 2023-07-17 10:50:15 +03:00
neon_arch
cff7de9d0f feat: rephrase the comment to break it into smaller sentences. 2023-07-17 10:47:29 +03:00
neon_arch
74e4fc6169 Merge branch 'rolling' into improve-async-multithreading 2023-07-15 19:50:31 +03:00
neon_arch
94ef62eec9 feat: add documentation to code 2023-07-15 13:36:46 +03:00
neon_arch
8133de1758 feat: add check to conditionaly pass empty_result_set to true to UI 2023-07-14 21:29:01 +03:00
neon_arch
a3edf70d1e feat: add code to pass engine errors to the UI 2023-07-14 21:27:23 +03:00
neon_arch
3aeb3b3e73 feat: add setter functions to SearchResults struct 2023-07-14 21:26:29 +03:00
neon_arch
9ee516e9b1 feat: fix code for clippy suggestions 2023-07-14 21:18:26 +03:00
neon_arch
3e6f7867eb feat: add code to parse cookies and handle engine selection from UI 2023-07-14 17:20:46 +03:00
neon_arch
27247baf0c feat: define common behaviour for all engines 2023-07-14 17:17:05 +03:00
neon_arch
c223fed394 feat: remove the time crate import 2023-07-14 17:16:13 +03:00
neon_arch
2f01651be0 feat: fix bugs and add code to handle engine selections
Co-authored-by: Sabrina Jewson <58880148+SabrinaJewson@users.noreply.github.com>
2023-07-14 12:56:06 +03:00
neon_arch
0781385393 feat: implement async multithreading and engine selection code 2023-07-11 19:44:38 +03:00
neon_arch
897ab0807f feat: fix the argument type used 2023-07-11 19:43:32 +03:00
neon_arch
f9b9e87a0e feat: rewrite code by implementing common engine trait SearchEngine 2023-07-11 19:42:17 +03:00
neon_arch
b72af01e0e feat: implement common engine trait 2023-07-11 19:41:34 +03:00
neon_arch
ff79c1fcfb feat: code to parse and handle the new config option 2023-07-11 19:38:59 +03:00
Trickshotblaster
99936bcb1a Rename getter functions 2023-07-04 15:11:30 -07:00
Milim
4cb38b1b54 Fix docs 2023-07-03 19:38:17 +02:00
Milim
440216871d Rename Things, refactor some code
BREAKING: renames `binding_ip_addr` to `binding_ip` and `redis_connection_url` to `redis_url`.

Renames a lot of internals as well, but they are to many to mention.
2023-07-03 19:30:25 +02:00
Milim
dbe5b533d6 fix spelling 2023-06-29 19:40:49 +02:00
Milim
e2907edf6e refactor config helper function 2023-06-29 19:10:09 +02:00
Milim
1f90b4e15c refactor search route and change default page to 0
that thing was utterly insane, and i am not sorry for saying this
2023-06-29 18:49:38 +02:00
xffxff
ebb9e9ee2b make format happy 2023-06-15 06:48:37 +08:00
zhou fan
b6faa09750
Update src/engines/engine_models.rs
Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
2023-06-15 06:28:21 +08:00
xffxff
1ebf888085 log the error from engines 2023-06-15 06:27:45 +08:00
xffxff
9a4cf949e4 improve error handling by using error-stack crate 2023-06-14 21:23:00 +08:00
neon_arch
c2280b7349 chore: add source and an appropriate message to different error variants 2023-06-04 11:56:07 +03:00
neon_arch
4460730ca6 chore: make clippy github action happy 2023-06-01 12:21:45 +03:00
neon_arch
975e8a480b
Merge branch 'rolling' into handle-reqwest-errors 2023-05-31 17:20:08 +00:00
neon_arch
5962cca294 chore: provide a better and more standardized way to handle engine errors 2023-05-31 19:54:51 +03:00
neon_arch
13632f1f99 feat: remove random delays when debug is set to true 2023-05-29 21:28:09 +03:00
neon_arch
60317a3b75 chore: make format ci happy 2023-05-29 21:13:07 +03:00
neon_arch
d4df90160d feat: add an option to enable/disable debug mode 2023-05-29 21:09:07 +03:00
neon_arch
8e7dc68d2d feat: add an option to enable or disable logs 2023-05-27 19:50:20 +03:00
neon_arch
dc3308cb5f chore: rename from theme to public 2023-05-25 11:50:37 +03:00
neon_arch
29456650f1 feat: add ability to put the themes folder on different paths 2023-05-24 17:37:41 +03:00
zhou fan
e6bb981133
Merge branch 'rolling' into ability-to-have-config-on-diff-paths 2023-05-24 17:10:07 +08:00
neon_arch
a7a28ed8c6 chore: make websurfx directory and config file names as constants 2023-05-24 12:01:36 +03:00
neon_arch
89796a054f feat: add ability to put config file on different paths 2023-05-23 23:47:36 +03:00
XFFXFF
ea013e718e make format happy 2023-05-23 10:00:35 +00:00
XFFXFF
9773cee38d change the 'aggregator' option to 'production_use' 2023-05-23 10:00:05 +00:00
XFFXFF
05272884ba supports the option to add a random delay 2023-05-23 10:00:03 +00:00
XFFXFF
cecffe4155 make format happy 2023-05-23 09:34:46 +00:00
neon_arch
6e19246cb2 fix: reduce the time to build user-agents 2023-05-22 13:07:37 +03:00
MD AL AMIN TALUKDAR
ae2fdb557c
Removed unnecessary comment 2023-05-20 08:01:24 +05:30
MD AL AMIN TALUKDAR
9a204b2f98 Fix page_url assignment in search route
The page_url variable in the search route was not being properly assigned in certain cases. This commit fixes the issue by ensuring that page_url is assigned the correct value based on the search parameters.

In the match expression, the conditions have been adjusted to correctly handle the page number and construct the appropriate page_url. This ensures that the generated URL for the search page is accurate and reflects the search query and page number.

This change improves the functionality and reliability of the search route by correctly setting the page_url variable based on the provided search parameters.
2023-05-19 17:13:11 +05:30
neon_arch
019b3322e7 chore: add enum to handle different reqwest errors & add timeout to requests 2023-05-16 12:22:00 +03:00
XFFXFF
3c7edb80d0 refactor: reduce connections created with RedisCache 2023-05-15 00:20:43 +00:00
neon_arch
4364310b24 add support for dockerized container hosting 2023-05-09 18:53:12 +03:00
neon_arch
0502a8f551 improve code to evade bot detection and closes #8 2023-05-08 11:47:50 +03:00
neon_arch
c5c1684cc1 optimise code for large scale server use and closes #7 2023-05-07 21:18:19 +03:00
neon_arch
c170de8194 add code to evade ip blocking, improve pagination code and fix documentation 2023-05-02 11:58:21 +03:00
neon_arch
71fae8b496 add more colorschemes, fix tests and improve documentation 2023-04-30 20:04:31 +03:00
neon_arch
4a505fb1d5 add missing documentation in config.lua and application source code 2023-04-30 19:24:16 +03:00
neon_arch
137c62ed5f replace commandline arguments with config.lua & add support for changing themes & coloschemes 2023-04-30 18:16:08 +03:00
neon_arch
cb1edad658 fixing documentation test in lib.rs and adding library documentation 2023-04-28 14:43:02 +03:00
XFFXFF
12bb0eefb8 add a integration test for index page 2023-04-28 10:39:58 +08:00
neon_arch
fc69acea8f Improving source code documentation. 2023-04-27 17:53:28 +03:00
neon_arch
ed13a16ec5 updating and improving README.org 2023-04-27 16:06:59 +03:00
neon_arch
28fee6b10a Adding minor change and implementing Cloning ability. 2023-04-26 17:46:49 +03:00
neon_arch
f94ac503e2 Refactoring code and separating code into files for better maintainability 2023-04-25 16:30:04 +03:00
XFFXFF
3060b01ed9 add logging middleware 2023-04-25 18:57:18 +08:00
neon_arch
0d1213a241 Adds temporary fixes for #1 2023-04-24 13:12:44 +03:00
neon_arch
15fc415301 initial commit 2023-04-22 14:35:07 +03:00