0
0
mirror of https://github.com/neon-mmd/websurfx.git synced 2024-11-24 23:18:22 -05:00
Commit Graph

12 Commits

Author SHA1 Message Date
neon_arch
991f3f59de
perf: several optimizations for improving the performance of the engine (#540)
* ♻️ refactor: initialize & store the config & cache structs as a constant (#486)
- initializes & stores the config & cache structs as a static constant.
- Pass the config & cache structs as a static reference to all the
  functions handling their respective route.

*  perf: replace hashmaps with vectors for fetching & aggregating results (#486)
- replace hashmaps with vectors for fetching, collecting & aggregating results as it tends to be contigous & cache efficient data structure.
- refactor & redesign algorithms for fetching & aggregating results
  centered around vectors in aggregate function.

*  build: add the future crate (#486)

*  perf: use `futureunordered` for collecting results fetched from the tokio spawn tasks (#486)
- using the `futureunordered` instead of vector for collecting results
  reduces the time it takes to fetch the results as the results do not
  need to come in specific order so any result that gets fetched first
  gets collected in the `futureunordered` type.

Co-authored-by: Spencerjibz <spencernajib2@gmail.com>

*  perf: initialize new async connections parallely using tokio spawn tasks (#486)

*  perf: initialize redis pipeline struct once with the default size of 3 (#486)

*  perf: reduce branch predictions by reducing conditional code branches (#486)

*  test(unit): provide unit test for the `get_safesearch_level` function (#486)

*  perf: reduce clones & use index based loop to improve search results filtering performance (#486)

* 🚨 fix(clippy): make clippy/format checks happy (#486)

* 🚨 fix(build): make the cargo build check happy (#486)

*  perf: reduce the amount of clones, to_owneds & to_strings (#486)

*  perf: use async crates & methods & make functions async (#486)

* 🔖 chore(release): bump the app version (#486)

---------

Co-authored-by: Spencerjibz <spencernajib2@gmail.com>
2024-03-11 12:01:30 +03:00
neon_arch
9a4e450766 test(integration): fix the integration by providing an extra function argument (#424) 2023-12-11 21:15:11 +03:00
Ashwin Vinod
e704c26ed3
♻️ Refactor cache system (#399)
* ♻️ Refactor cache system

* 🐛 Fix cache not getting set

This patch also makes it that cookies are eagerly evaluated. This is
done to figure out the safe search level set by the user. The
performance hit wouldn't be much of a deal as the cookie is a small
json string

* 🔖 chore: bump the app version (#399)

* 🔖 chore: bump the app version (#399)

---------

Co-authored-by: alamin655 <mdalamin655@outlook.com>
2023-11-28 09:17:35 +03:00
neon_arch
47905f1e22 test: update the integration with the new changes (#302) 2023-11-18 21:29:35 +03:00
neon_arch
d33129c4c9 🧹 chore: make clippy happy (#244) 2023-09-17 19:56:48 +03:00
Zsombor Gegesy
996ff84c5b Cache refactor - add an in-memory cache, so redis is not needed 2023-09-15 23:21:19 +02:00
Milim
15dfda6ea9 Improve Aggregation function & config parser
Refactor aggregation function
Rename visiting_url to url, as they are always the same (see upstream engine scalping).
Refactor parsing function to be more readable.
2023-08-17 22:48:20 +02:00
neon_arch
4b4dc28cd2 feat: add control check to avoid reinitializing logging & fix tests 2023-08-03 00:12:09 +03:00
Milim
440216871d Rename Things, refactor some code
BREAKING: renames `binding_ip_addr` to `binding_ip` and `redis_connection_url` to `redis_url`.

Renames a lot of internals as well, but they are to many to mention.
2023-07-03 19:30:25 +02:00
neon_arch
c170de8194 add code to evade ip blocking, improve pagination code and fix documentation 2023-05-02 11:58:21 +03:00
neon_arch
71fae8b496 add more colorschemes, fix tests and improve documentation 2023-04-30 20:04:31 +03:00
XFFXFF
12bb0eefb8 add a integration test for index page 2023-04-28 10:39:58 +08:00