0
0
mirror of https://github.com/neon-mmd/websurfx.git synced 2024-11-27 00:08:22 -05:00
Commit Graph

82 Commits

Author SHA1 Message Date
neon_arch
991f3f59de
perf: several optimizations for improving the performance of the engine (#540)
* ♻️ refactor: initialize & store the config & cache structs as a constant (#486)
- initializes & stores the config & cache structs as a static constant.
- Pass the config & cache structs as a static reference to all the
  functions handling their respective route.

*  perf: replace hashmaps with vectors for fetching & aggregating results (#486)
- replace hashmaps with vectors for fetching, collecting & aggregating results as it tends to be contigous & cache efficient data structure.
- refactor & redesign algorithms for fetching & aggregating results
  centered around vectors in aggregate function.

*  build: add the future crate (#486)

*  perf: use `futureunordered` for collecting results fetched from the tokio spawn tasks (#486)
- using the `futureunordered` instead of vector for collecting results
  reduces the time it takes to fetch the results as the results do not
  need to come in specific order so any result that gets fetched first
  gets collected in the `futureunordered` type.

Co-authored-by: Spencerjibz <spencernajib2@gmail.com>

*  perf: initialize new async connections parallely using tokio spawn tasks (#486)

*  perf: initialize redis pipeline struct once with the default size of 3 (#486)

*  perf: reduce branch predictions by reducing conditional code branches (#486)

*  test(unit): provide unit test for the `get_safesearch_level` function (#486)

*  perf: reduce clones & use index based loop to improve search results filtering performance (#486)

* 🚨 fix(clippy): make clippy/format checks happy (#486)

* 🚨 fix(build): make the cargo build check happy (#486)

*  perf: reduce the amount of clones, to_owneds & to_strings (#486)

*  perf: use async crates & methods & make functions async (#486)

* 🔖 chore(release): bump the app version (#486)

---------

Co-authored-by: Spencerjibz <spencernajib2@gmail.com>
2024-03-11 12:01:30 +03:00
ddotthomas
41ab8a2a76
config option to configure the https adaptive window size for requests (#529)
* Added config option to enable the reqwest client adaptive window

* Change adaptive window config name

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* Modified documentation

* Trimmed down aggregate parameters

---------

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
2024-02-28 15:08:29 +03:00
Spencerjibz
6b9469e4b3 cache the next, current and previous results in a separate task 2024-01-22 19:29:08 +00:00
ddotthomas
31c9c676ee Change get_safe_search_level logic back 2024-01-12 09:51:31 -07:00
ddotthomas
ca96a76958 change: renamed get_safesearch_level variables 2024-01-11 11:58:09 -07:00
ddotthomas
80e950de3b refactor: changed Cookie to use Cow to facilitate using references when building 2024-01-10 14:14:14 -07:00
ddotthomas
d912bff94e change: revert Cookie Strings back to &str 2024-01-08 11:46:21 -07:00
ddotthomas
6e9250c03a fix: safe search url parameter ignored 2024-01-08 09:19:36 -07:00
ddotthomas
7d762b3726 refactor: moved settings parsing out of results 2024-01-07 16:29:39 -07:00
ddotthomas
dde117e7e6 modify: changed cache_key format data 2024-01-04 11:05:09 -07:00
ddotthomas
86b0d3d6c9 Moved parsing of cookie_value, config to cache_key 2024-01-02 16:26:04 -07:00
neon_arch
36e2ac93be
Merge branch 'rolling' into FIX/463_results_from_different_search_engines_get_cached_as_the_same_key 2024-01-02 12:29:54 +03:00
ddotthomas
db1115f19e fix: add comma separator and format macro 2024-01-01 19:36:18 -07:00
Леонтий Вартанян
5020f36c90
♻️ refactor: standardize the content-type header by using an enum value over typing it manually (#474)
* ♻️  refactor: change content-type

* 🐛  fix: change parameters that were passed to the settings function

---------

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
2023-12-31 13:31:07 +03:00
ddotthomas
4f4cb220cc fix: changed dereferencing 2023-12-30 22:18:15 -07:00
ddotthomas
286bcf1bd3 fix: added a config parse backup 2023-12-30 22:12:43 -07:00
ddotthomas
bfeb81270c fix: modify cache key with engines used for search 2023-12-30 19:19:22 -07:00
alamin655
7f84c6346d
Merge branch 'rolling' into CHORE/461_display-the-user-provided-settings-from-the-config-or-the-ui-in-the-settings-page 2023-12-30 12:23:47 +05:30
ddotthomas
d073aa247a
♻️ refactor: start tracking page numbers from 0 instead of 1 in the backend (#467)
Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
2023-12-29 21:20:38 +03:00
neon_arch
d8943709c7 🚨 chore: make cargo format checks happy (#461) 2023-12-28 21:08:42 +03:00
neon_arch
93afb6b8c9
Merge branch 'rolling' into CHORE/461_display-the-user-provided-settings-from-the-config-or-the-ui-in-the-settings-page 2023-12-28 21:04:21 +03:00
neon_arch
26aa345f06 🚸 chore: pass the safe_search_level & upstream_search_engines as a hashmap in the settings view function (#461) 2023-12-28 19:58:18 +03:00
maokwen
9f5213cf42
🐛 fix: explicit content-type header for the search page response (#458) 2023-12-28 07:05:21 +03:00
maokwen
43357493d5
🐛 fix: explicit Content-Type header for HTTP response (#457) 2023-12-27 11:24:44 +03:00
neon_arch
1a2a833597 feat: pass the new animation config option (#424) 2023-12-12 15:04:44 +03:00
Ashwin Vinod
e704c26ed3
♻️ Refactor cache system (#399)
* ♻️ Refactor cache system

* 🐛 Fix cache not getting set

This patch also makes it that cookies are eagerly evaluated. This is
done to figure out the safe search level set by the user. The
performance hit wouldn't be much of a deal as the cookie is a small
json string

* 🔖 chore: bump the app version (#399)

* 🔖 chore: bump the app version (#399)

---------

Co-authored-by: alamin655 <mdalamin655@outlook.com>
2023-11-28 09:17:35 +03:00
víctor
5e2669b6de
♻️ Clean some code, fix bug (#396)
* ♻️ Improve src/handler

Removes unnecessary submoduling & adjusts some weird code

* ♻️ Cleaner code

* 🐛 Fixed issue where code would overflow if page=0
2023-11-27 09:45:28 +03:00
neon_arch
03d649b97b 🔥 chore: remove the debug macro from the code (#302) 2023-11-18 21:51:21 +03:00
neon_arch
eee2f110b8 🚨 chore: make cargo clippy linter happy (#302) 2023-11-18 21:50:37 +03:00
neon_arch
38ba4bd6cb ♻️ refactor(routes): serve the new maud file for search page and remove unwanted functions and pass values to the maud html template directly (#302) 2023-11-17 22:16:17 +03:00
neon_arch
64c4d2c23a ♻️ refactor(routes): serve new the maud files instead of handlebars files (#302) 2023-11-17 22:12:25 +03:00
Zsombor Gegesy
75a77d25f0 Create separate search_result_parser 2023-10-07 00:33:44 +02:00
neon_arch
3d489208db 🧹 chore: make github actions happy (#227) 2023-09-23 15:03:48 +03:00
alamin655
017036ed00
Merge branch 'rolling' into feat-inform-user-when-no-engines-are-selected 2023-09-23 17:12:58 +05:30
neon_arch
173c6ba821 feat: add code to handle aggregation when no engines are selected (#227) 2023-09-23 12:50:36 +03:00
neon_arch
72fec47892 feat: implement code to fetch results based on parameters, cookies & config (#210) 2023-09-22 19:54:26 +03:00
neon_arch
fcfd112e59 🛠️ fix: improve the documentation & move code in the correct files (#244) 2023-09-17 12:51:32 +03:00
Zsombor Gegesy
320f5f4720 Remove unnecessary clones 2023-09-15 23:21:19 +02:00
Zsombor Gegesy
519ebe0fd8 Instead of caching jsons, we can cache the original structure 2023-09-15 23:21:19 +02:00
Zsombor Gegesy
996ff84c5b Cache refactor - add an in-memory cache, so redis is not needed 2023-09-15 23:21:19 +02:00
neon_arch
ca4447fcdf Merge branch 'rolling' into reorganize-code-and-restructure-the-codebase 2023-09-13 18:31:37 +03:00
neon_arch
c60fdb8366 🧹 chore: make github actions happy (#205) 2023-09-12 18:53:32 +03:00
neon_arch
1a222217c4 🧹 chore: make github actions happy (#205) 2023-09-12 17:59:33 +03:00
neon_arch
fb231de416
Merge branch 'rolling' into change-document-style-with-linter-warnings 2023-09-12 17:49:46 +03:00
neon_arch
12bfc5276a 🧹 chore: make clippy happy (#201) 2023-09-10 18:56:54 +03:00
alamin655
60b2fcc27a
Merge branch 'rolling' into feat-disallow-user-to-search-via-lists 2023-09-10 20:51:10 +05:30
neon_arch
82a78e3b42
Merge branch 'rolling' into optimize-and-make-code-idiomatic 2023-09-10 18:11:12 +03:00
neon_arch
8dfe2e1b2d
Merge branch 'rolling' into optimize-and-make-code-idiomatic-3 2023-09-10 17:58:28 +03:00
neon_arch
493c56bd02 ⚙️ refactor: reorganize code & restructure codebase for better maintainability (#207) 2023-09-03 20:50:50 +03:00
neon_arch
049b1c1ddd ⚙️ refactor: change & add documentation to the code based on the lints (#205) 2023-09-03 19:23:34 +03:00