0
0
mirror of https://github.com/neon-mmd/websurfx.git synced 2024-11-22 14:08:23 -05:00
Commit Graph

23 Commits

Author SHA1 Message Date
neon_arch
991f3f59de
perf: several optimizations for improving the performance of the engine (#540)
* ♻️ refactor: initialize & store the config & cache structs as a constant (#486)
- initializes & stores the config & cache structs as a static constant.
- Pass the config & cache structs as a static reference to all the
  functions handling their respective route.

*  perf: replace hashmaps with vectors for fetching & aggregating results (#486)
- replace hashmaps with vectors for fetching, collecting & aggregating results as it tends to be contigous & cache efficient data structure.
- refactor & redesign algorithms for fetching & aggregating results
  centered around vectors in aggregate function.

*  build: add the future crate (#486)

*  perf: use `futureunordered` for collecting results fetched from the tokio spawn tasks (#486)
- using the `futureunordered` instead of vector for collecting results
  reduces the time it takes to fetch the results as the results do not
  need to come in specific order so any result that gets fetched first
  gets collected in the `futureunordered` type.

Co-authored-by: Spencerjibz <spencernajib2@gmail.com>

*  perf: initialize new async connections parallely using tokio spawn tasks (#486)

*  perf: initialize redis pipeline struct once with the default size of 3 (#486)

*  perf: reduce branch predictions by reducing conditional code branches (#486)

*  test(unit): provide unit test for the `get_safesearch_level` function (#486)

*  perf: reduce clones & use index based loop to improve search results filtering performance (#486)

* 🚨 fix(clippy): make clippy/format checks happy (#486)

* 🚨 fix(build): make the cargo build check happy (#486)

*  perf: reduce the amount of clones, to_owneds & to_strings (#486)

*  perf: use async crates & methods & make functions async (#486)

* 🔖 chore(release): bump the app version (#486)

---------

Co-authored-by: Spencerjibz <spencernajib2@gmail.com>
2024-03-11 12:01:30 +03:00
Evan Yang
efa8efc6c7
Compression and encryption for the cached search results (#443)
* attempt1

* rough draft

* add features and their optional dependancies

* add encryption and compression error variants

* add  a sample implementation to cache trait

* Update src/cache/cacher.rs

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* adjust comment so feature flag would apply?

* adjust feature flag so it applies?

* formatting

* Update src/cache/cacher.rs

update documentation

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* [features]Add base64 and chacha20 dependencies for compress-cache-results and encrypt-cache-results

* move encryption key and cipher logic to separate sub module

* added cacha20 and cec-results feature

* added cacha20 and cec-results feature

* added compression and encryption helper functions to trait implementations

* added compression and encryption  implementation for inMemoryCache

* base64 is only requried when redis-cache feature is enabled

* add error case for base64 and encryption/compression implementation to redisCache

* Refactor cacher to remove regex dependency

* fmt cache error and cacher

* Update Cargo.toml

disabling the unneeded default-features

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* fix unused import warning for mimalloc

* remove deprecated method

* add doc comments for encryption module

* fix known bugs and use cfg-if module

* make cfg-if an optional dependency

* use feature-flag instead of maco lint

* add comment to explain type complexity

* bump app version

* Update src/cache/encryption.rs

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* fixed type complexity and add docs for types

---------

Co-authored-by: Spencer Najib <spencernajib2@gmail.com>
Co-authored-by: alamin655 <mdalamin655@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
Co-authored-by: Spencerjibz <=spencernajib2@gmail.com>
Co-authored-by: spencer <spencer@DESKTOP-SIF13AR>
2024-01-11 11:10:35 +00:00
Ashwin Vinod
e704c26ed3
♻️ Refactor cache system (#399)
* ♻️ Refactor cache system

* 🐛 Fix cache not getting set

This patch also makes it that cookies are eagerly evaluated. This is
done to figure out the safe search level set by the user. The
performance hit wouldn't be much of a deal as the cookie is a small
json string

* 🔖 chore: bump the app version (#399)

* 🔖 chore: bump the app version (#399)

---------

Co-authored-by: alamin655 <mdalamin655@outlook.com>
2023-11-28 09:17:35 +03:00
Zsombor Gegesy
76795c43cc Make the cache types compile time feature flags too, and make it more configurable! 2023-09-15 23:21:19 +02:00
Zsombor Gegesy
996ff84c5b Cache refactor - add an in-memory cache, so redis is not needed 2023-09-15 23:21:19 +02:00
neon_arch
028463bb36 ⚙️ refactor: replace jemalloc with mimalloc (#178)(#180) 2023-08-29 20:07:28 +03:00
neon_arch
fceacdb916 🧹 chore: make cargo format happy (#180)(#178) 2023-08-27 21:17:21 +03:00
neon_arch
4afc0d16f0 ⚙️ refactor: add dhat profiler configuration with feat flag (#180)(#178) 2023-08-27 20:46:19 +03:00
Milim
87e230d731 fix debug level not being set properly 2023-08-19 09:09:09 +02:00
neon_arch
653d08c801 🔧 chore: fix link in the logs & add logs for engine error (#170) 2023-08-05 19:16:27 +03:00
neon_arch
4b4dc28cd2 feat: add control check to avoid reinitializing logging & fix tests 2023-08-03 00:12:09 +03:00
neon_arch
d8bd2feb6e feat: add new config option to manage threads and improve logging 2023-08-02 20:05:39 +03:00
Milim
440216871d Rename Things, refactor some code
BREAKING: renames `binding_ip_addr` to `binding_ip` and `redis_connection_url` to `redis_url`.

Renames a lot of internals as well, but they are to many to mention.
2023-07-03 19:30:25 +02:00
neon_arch
60317a3b75 chore: make format ci happy 2023-05-29 21:13:07 +03:00
neon_arch
d4df90160d feat: add an option to enable/disable debug mode 2023-05-29 21:09:07 +03:00
neon_arch
8e7dc68d2d feat: add an option to enable or disable logs 2023-05-27 19:50:20 +03:00
neon_arch
137c62ed5f replace commandline arguments with config.lua & add support for changing themes & coloschemes 2023-04-30 18:16:08 +03:00
XFFXFF
12bb0eefb8 add a integration test for index page 2023-04-28 10:39:58 +08:00
neon_arch
fc69acea8f Improving source code documentation. 2023-04-27 17:53:28 +03:00
neon_arch
f94ac503e2 Refactoring code and separating code into files for better maintainability 2023-04-25 16:30:04 +03:00
XFFXFF
3060b01ed9 add logging middleware 2023-04-25 18:57:18 +08:00
neon_arch
0d1213a241 Adds temporary fixes for #1 2023-04-24 13:12:44 +03:00
neon_arch
15fc415301 initial commit 2023-04-22 14:35:07 +03:00