0
0
mirror of https://github.com/neon-mmd/websurfx.git synced 2024-11-24 06:58:22 -05:00
Commit Graph

37 Commits

Author SHA1 Message Date
Kekma
2a4dd07752
feat: config option to configure the number of connections to have in the reqwest pool (#552)
* 🔧 Config: Updated `number_of_https_connections` for the HTTPS connection pool.

* (feat) : added the max idle pool per host option

* 📝 docs(config): add documentation back to the `http_adaptive_window_size` option (#532)

---------

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-09-30 15:22:25 +03:00
neon_arch
39af9096ef perf: replace Vec<T> with Box<T> & initialize vectors with capacity by default (#603) 2024-09-05 22:03:23 +05:30
neon_arch
4bb6c5e90b perf: initialize vectors with capacity by default & use Arc<T> to partially clone memory cache struct (#603) 2024-09-05 21:57:18 +05:30
neon_arch
acee5d892d 🐛 fix: replace deprecated set_ex command with set_options in cache_json function (#592) 2024-09-02 21:17:31 +05:30
neon_arch
991f3f59de
perf: several optimizations for improving the performance of the engine (#540)
* ♻️ refactor: initialize & store the config & cache structs as a constant (#486)
- initializes & stores the config & cache structs as a static constant.
- Pass the config & cache structs as a static reference to all the
  functions handling their respective route.

*  perf: replace hashmaps with vectors for fetching & aggregating results (#486)
- replace hashmaps with vectors for fetching, collecting & aggregating results as it tends to be contigous & cache efficient data structure.
- refactor & redesign algorithms for fetching & aggregating results
  centered around vectors in aggregate function.

*  build: add the future crate (#486)

*  perf: use `futureunordered` for collecting results fetched from the tokio spawn tasks (#486)
- using the `futureunordered` instead of vector for collecting results
  reduces the time it takes to fetch the results as the results do not
  need to come in specific order so any result that gets fetched first
  gets collected in the `futureunordered` type.

Co-authored-by: Spencerjibz <spencernajib2@gmail.com>

*  perf: initialize new async connections parallely using tokio spawn tasks (#486)

*  perf: initialize redis pipeline struct once with the default size of 3 (#486)

*  perf: reduce branch predictions by reducing conditional code branches (#486)

*  test(unit): provide unit test for the `get_safesearch_level` function (#486)

*  perf: reduce clones & use index based loop to improve search results filtering performance (#486)

* 🚨 fix(clippy): make clippy/format checks happy (#486)

* 🚨 fix(build): make the cargo build check happy (#486)

*  perf: reduce the amount of clones, to_owneds & to_strings (#486)

*  perf: use async crates & methods & make functions async (#486)

* 🔖 chore(release): bump the app version (#486)

---------

Co-authored-by: Spencerjibz <spencernajib2@gmail.com>
2024-03-11 12:01:30 +03:00
abdulahad5112
fbf73634ee
🐛 Undeclared mini-mocha crate error when building the app with features other than memory-cache #493 (#501) 2024-01-30 07:09:33 +00:00
Spencerjibz
51214dc23a add support for caching N number of key-values pairs instead of only 3 2024-01-24 22:56:11 +00:00
Spencerjibz
f5cf5f9151 changed cache_results method to support multiple values 2024-01-22 19:26:35 +00:00
Spencerjibz
c762f9cf8e use redis pipeline to set multiple values at once 2024-01-22 19:20:16 +00:00
Evan Yang
efa8efc6c7
Compression and encryption for the cached search results (#443)
* attempt1

* rough draft

* add features and their optional dependancies

* add encryption and compression error variants

* add  a sample implementation to cache trait

* Update src/cache/cacher.rs

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* adjust comment so feature flag would apply?

* adjust feature flag so it applies?

* formatting

* Update src/cache/cacher.rs

update documentation

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* [features]Add base64 and chacha20 dependencies for compress-cache-results and encrypt-cache-results

* move encryption key and cipher logic to separate sub module

* added cacha20 and cec-results feature

* added cacha20 and cec-results feature

* added compression and encryption helper functions to trait implementations

* added compression and encryption  implementation for inMemoryCache

* base64 is only requried when redis-cache feature is enabled

* add error case for base64 and encryption/compression implementation to redisCache

* Refactor cacher to remove regex dependency

* fmt cache error and cacher

* Update Cargo.toml

disabling the unneeded default-features

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* fix unused import warning for mimalloc

* remove deprecated method

* add doc comments for encryption module

* fix known bugs and use cfg-if module

* make cfg-if an optional dependency

* use feature-flag instead of maco lint

* add comment to explain type complexity

* bump app version

* Update src/cache/encryption.rs

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* fixed type complexity and add docs for types

---------

Co-authored-by: Spencer Najib <spencernajib2@gmail.com>
Co-authored-by: alamin655 <mdalamin655@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
Co-authored-by: Spencerjibz <=spencernajib2@gmail.com>
Co-authored-by: spencer <spencer@DESKTOP-SIF13AR>
2024-01-11 11:10:35 +00:00
Ashwin Vinod
5a8d61f231
Config option to customize the cache invalidation/expiry time (#403) 2023-11-30 15:09:17 +03:00
Ashwin Vinod
e704c26ed3
♻️ Refactor cache system (#399)
* ♻️ Refactor cache system

* 🐛 Fix cache not getting set

This patch also makes it that cookies are eagerly evaluated. This is
done to figure out the safe search level set by the user. The
performance hit wouldn't be much of a deal as the cookie is a small
json string

* 🔖 chore: bump the app version (#399)

* 🔖 chore: bump the app version (#399)

---------

Co-authored-by: alamin655 <mdalamin655@outlook.com>
2023-11-28 09:17:35 +03:00
alamin655
abc59b2858
🧹 chore: make clippy happy (#386) 2023-11-20 21:38:20 +05:30
alamin655
fc830c4683
Update redis_cacher.rs 2023-11-20 21:09:27 +05:30
S Dheeraj
89542072c8
Updated redis_cacher.rs 2023-11-20 08:53:30 +05:30
S Dheeraj
dc5fa842c0 #385 issue resolved
replaced md5 hashing algorithm with sha-256 hashing algorithm
2023-11-19 22:09:34 +05:30
neon_arch
d33129c4c9 🧹 chore: make clippy happy (#244) 2023-09-17 19:56:48 +03:00
neon_arch
578c7bcf77 🛠️ fix: improve the documentation for the code (#244) 2023-09-17 12:48:11 +03:00
neon_arch
03384d4e04 🛠️ fix: implement hybrid caching and improve documentation (#244) 2023-09-17 12:47:02 +03:00
Zsombor Gegesy
e69126c5ea Fix git rebase problems, and the failing doctest 2023-09-15 23:25:17 +02:00
Zsombor Gegesy
1e7805cf42 Rename features, make the memory-cache the default 2023-09-15 23:21:19 +02:00
Zsombor Gegesy
76795c43cc Make the cache types compile time feature flags too, and make it more configurable! 2023-09-15 23:21:19 +02:00
Zsombor Gegesy
320f5f4720 Remove unnecessary clones 2023-09-15 23:21:19 +02:00
Zsombor Gegesy
519ebe0fd8 Instead of caching jsons, we can cache the original structure 2023-09-15 23:21:19 +02:00
Zsombor Gegesy
996ff84c5b Cache refactor - add an in-memory cache, so redis is not needed 2023-09-15 23:21:19 +02:00
neon_arch
1a222217c4 🧹 chore: make github actions happy (#205) 2023-09-12 17:59:33 +03:00
neon_arch
fb231de416
Merge branch 'rolling' into change-document-style-with-linter-warnings 2023-09-12 17:49:46 +03:00
neon_arch
049b1c1ddd ⚙️ refactor: change & add documentation to the code based on the lints (#205) 2023-09-03 19:23:34 +03:00
neon_arch
1de52decd3 ⚙️ refactor: add clone trait to RedisCache struct (#180)(#178) 2023-08-29 20:10:32 +03:00
neon_arch
5f1a43976f ⚙️ refactor: add error.rs module (#180)(#178) 2023-08-27 20:53:55 +03:00
neon_arch
01d8c7ae4c ⚙️ refactor: add new pooling error type for pooling code (#180)(#178) 2023-08-27 20:52:16 +03:00
neon_arch
db93c31603 ⚙️ refactor: implement async pooling for redis connections (#180)(#178) 2023-08-27 20:50:42 +03:00
Trickshotblaster
99936bcb1a Rename getter functions 2023-07-04 15:11:30 -07:00
Milim
440216871d Rename Things, refactor some code
BREAKING: renames `binding_ip_addr` to `binding_ip` and `redis_connection_url` to `redis_url`.

Renames a lot of internals as well, but they are to many to mention.
2023-07-03 19:30:25 +02:00
XFFXFF
cecffe4155 make format happy 2023-05-23 09:34:46 +00:00
XFFXFF
3c7edb80d0 refactor: reduce connections created with RedisCache 2023-05-15 00:20:43 +00:00
neon_arch
c170de8194 add code to evade ip blocking, improve pagination code and fix documentation 2023-05-02 11:58:21 +03:00