0
0
mirror of https://github.com/neon-mmd/websurfx.git synced 2024-11-23 14:38:21 -05:00
Commit Graph

28 Commits

Author SHA1 Message Date
Spencer
bb50e8bb25
Ranking of aggregated search results based on relevancy of the search result to the user's search query (#549)
* add sorting by relevance and merge new changes

* fix conflicts

* Update src/models/aggregation_models.rs

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* Update src/models/aggregation_models.rs

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* Update Cargo.toml

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* Update Cargo.toml

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* Update Cargo.toml

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

* enable non-static-synonyms features

---------

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
2024-03-25 09:16:49 +00:00
neon_arch
991f3f59de
perf: several optimizations for improving the performance of the engine (#540)
* ♻️ refactor: initialize & store the config & cache structs as a constant (#486)
- initializes & stores the config & cache structs as a static constant.
- Pass the config & cache structs as a static reference to all the
  functions handling their respective route.

*  perf: replace hashmaps with vectors for fetching & aggregating results (#486)
- replace hashmaps with vectors for fetching, collecting & aggregating results as it tends to be contigous & cache efficient data structure.
- refactor & redesign algorithms for fetching & aggregating results
  centered around vectors in aggregate function.

*  build: add the future crate (#486)

*  perf: use `futureunordered` for collecting results fetched from the tokio spawn tasks (#486)
- using the `futureunordered` instead of vector for collecting results
  reduces the time it takes to fetch the results as the results do not
  need to come in specific order so any result that gets fetched first
  gets collected in the `futureunordered` type.

Co-authored-by: Spencerjibz <spencernajib2@gmail.com>

*  perf: initialize new async connections parallely using tokio spawn tasks (#486)

*  perf: initialize redis pipeline struct once with the default size of 3 (#486)

*  perf: reduce branch predictions by reducing conditional code branches (#486)

*  test(unit): provide unit test for the `get_safesearch_level` function (#486)

*  perf: reduce clones & use index based loop to improve search results filtering performance (#486)

* 🚨 fix(clippy): make clippy/format checks happy (#486)

* 🚨 fix(build): make the cargo build check happy (#486)

*  perf: reduce the amount of clones, to_owneds & to_strings (#486)

*  perf: use async crates & methods & make functions async (#486)

* 🔖 chore(release): bump the app version (#486)

---------

Co-authored-by: Spencerjibz <spencernajib2@gmail.com>
2024-03-11 12:01:30 +03:00
Jann Marc Villablanca
669e365913
feat: add new helper function to fetch upstream search engine JSON response (#504)
Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
2024-01-30 13:37:50 +00:00
ddotthomas
80e950de3b refactor: changed Cookie to use Cow to facilitate using references when building 2024-01-10 14:14:14 -07:00
ddotthomas
d912bff94e change: revert Cookie Strings back to &str 2024-01-08 11:46:21 -07:00
ddotthomas
7d762b3726 refactor: moved settings parsing out of results 2024-01-07 16:29:39 -07:00
Jann Marc Villablanca
50aa52c485
Bing for the search engine (#473) (#473)
fix: add closing curly brace

- accidentally removed from previous PR. i hope i did not ruin some
  build

* feat: implement mojeek engine

* Merge branch 'rolling' into FEAT_316_bing_search_engine

*  feat: implement bing engine

* Fix: include cookie string to header_map

* Fix: include <strong> tags from upstream search

* Merge branch 'rolling' into FEAT_316_bing_search_engine

Co-authored-by: neon-mmd <132049916+neon-mmd@users.noreply.github.com>
2024-01-01 14:57:31 +03:00
Jann Marc Villablanca
9f23a1c70b
feat(engine): mojeek for the search engine (#464)
* 🧑‍💻 fix: add closing curly brace in the nix build step

*  feat: add code to provide search results from `mojeek` engine

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-12-29 19:21:06 +03:00
neon_arch
b00f76627b
Merge branch 'rolling' into FEAT/424_reduce-animations-effects-support-for-the-themes 2023-12-12 15:24:42 +03:00
neon_arch
1c5a317c4d feat: add a new field animation to the struct style (#424) 2023-12-11 12:27:29 +03:00
alamin655
c02006c297
LibreX for the search engine (#429)
*  feat(engine): provide librex search engine (#318)

*  feat(engine): provide librex search engine (#318)

*  feat(engine): provide librex search engine (#318)

* 🔖 chore(release): bump the app version (#318)

* 🔖 chore(release): bump the app version (#318)

*  feat(engine): provide librex search engine (#318)

*  feat(engine): provide librex search engine (#429)

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

*  feat(engine): provide librex search engine (#429)

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

*  feat(engine): provide librex search engine (#429)

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

*  feat(engine): provide librex search engine (#429)

*  feat(engine): provide librex search engine (#429)

*  feat(engine): provide librex search engine (#429)

*  feat(engine): provide librex search engine (#429)

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>

---------

Co-authored-by: neon_arch <mustafadhuleb53@gmail.com>
2023-12-09 14:25:28 +03:00
neon_arch
e1e426c517 feat(engine): provide startpage search engine (#314) 2023-12-05 20:47:28 +03:00
alamin655
ae9fa5b388
Merge branch 'rolling' into PERF/384_optimize-the-performance-of-fetching-results-in-the-websurfx-search-engine-backend 2023-11-20 21:03:12 +05:30
neon_arch
b42adaa5a3 ️ perf: rewrite the code by using a constant storing a prebuilt client globally for each thread (#384) 2023-11-20 15:27:49 +03:00
neon_arch
686e26ad6d ⚰️ chore: remove the serialize & deserialize traits on the struct (#302) 2023-11-18 21:43:28 +03:00
neon_arch
141ae26066 ♻️ refactor: remove the code to add the query & style in the SearchResults struct & also remove the associated fields from the struct (#302) 2023-11-18 21:38:02 +03:00
jkaczmarkiewicz
27bc52c008
Brave for the search engine (#335)
* feat: implement brave engine

* refactor: correct indentations in stylelint config

* docs: add dummy config option to config.lua

* feat: implement safe_search_level in brave engine

* refactor: move var to format

* fix: make strict search above level 1
2023-10-16 20:25:15 +03:00
Zsombor Gegesy
f56002dca6 Rename the error to NoSuchEngineFound and add the name of missing engine to it 2023-10-08 22:30:42 +02:00
Zsombor Gegesy
57c73d38c8 Refactor the search result parsing 2023-10-07 00:33:44 +02:00
Zsombor Gegesy
75a77d25f0 Create separate search_result_parser 2023-10-07 00:33:44 +02:00
alamin655
017036ed00
Merge branch 'rolling' into feat-inform-user-when-no-engines-are-selected 2023-09-23 17:12:58 +05:30
neon_arch
b428cedd7a feat: add new check value for no engine selected (#227) 2023-09-23 12:48:01 +03:00
neon_arch
d2e482251e feat: add the safe search field in the Cookie struct (#210) 2023-09-22 19:53:34 +03:00
neon_arch
5c8cbee5d9 feat: implement code to pass safe search option in the json results (#210) 2023-09-22 19:51:10 +03:00
neon_arch
8c239e2313 🛠️ fix: make the redis_url option only available on redis-cache feature (#244) 2023-09-17 12:50:25 +03:00
Zsombor Gegesy
519ebe0fd8 Instead of caching jsons, we can cache the original structure 2023-09-15 23:21:19 +02:00
neon_arch
ca4447fcdf Merge branch 'rolling' into reorganize-code-and-restructure-the-codebase 2023-09-13 18:31:37 +03:00
neon_arch
493c56bd02 ⚙️ refactor: reorganize code & restructure codebase for better maintainability (#207) 2023-09-03 20:50:50 +03:00