664 Commits (f645a96d18efc9fb014cf75a8f21864becb943ec)

Author SHA1 Message Date
Barış Soner Uşaklı 4d9bc30d1f
feat: async3 upgrade (#7639)
* feat: async3 upgrade WIP

* fix: async.doWhilst

* fix: async early exit

* fix: psql doUntil

* fix: psql again
6 years ago
Barış Soner Uşaklı 2c98dd5f9d
Object cache refactor (#7558)
* fix: cache refactor

db.getObjectField no longer loads entire object
db.getObjectsFields only clones data once
more tests

* feat: add back cache to redis

db.getObjectField no longer loads entire object
6 years ago
Barış Soner Uşaklı 4df925e76a fix: remove redis object cache 6 years ago
Baris Usakli 45d7e37ae8 fix: tests 6 years ago
Baris Usakli b48b6d75e5 fix: callbacks 6 years ago
Baris Usakli 7addc5a0e2 fix: return early if keys is empty 6 years ago
Baris Usakli b3962c1694 fix: return early if keys is empty 6 years ago
Barış Soner Uşaklı 5917dec288
fix: only allow numbers as scores (#7356)
* zadd score checks

* fix: only allow numbers as scores

* fix: convert values to strings
6 years ago
Barış Soner Uşaklı 2e2c3ac110 fix: #7352 6 years ago
Barış Soner Uşaklı fab52b840c fix: prevent crash if results.stats or results.serverStatus is undefined 6 years ago
Barış Soner Uşaklı a22a3a9861 fix: log error to prevent headers already sent
retry on uniq index fail
6 years ago
Peter Jaszkowiak e1ca2d81f9 refactor: use loash when possible (#7230) 6 years ago
Barış Soner Uşaklı 3917022a48 fix: #7139 6 years ago
Barış Soner Uşaklı fef7e13c9e feat: small fixes 6 years ago
Barış Soner Uşaklı d2c2d56f59 feat: change sortedSetsScore 6 years ago
Barış Soner Uşaklı 29b63ae7fb fix: #6979 6 years ago
Barış Soner Uşaklı 00afc5b3ab fix: remove log 6 years ago
Barış Soner Uşaklı eb7ae54f81
Category watch state (#7109)
* feat: wip, category watch change

* feat: pass data to client

* feat: allow changing state

* fix: account page categories

* fix: show in unread if topic is followed or category is watched

* feat: add default watch state to acp

* feat: save user category watch state

* feat: update unread recent pages

* fix: remove dupe code

* fix: flip conditions

* fix: handle empty arrays

* fix: ignore/watch on others profile

* feat: upgrade script for category states

if there are any users ignoring categories set their state in new zset and delete cid:<cid>:ignorers

* fix: upgrade

* fix: tests

* fix: redis count

* fix: more tests
6 years ago
Barış Soner Uşaklı d117df77f7 fix: #7102 6 years ago
Barış Soner Uşaklı 85a07e999b fix: #7102 6 years ago
Barış Soner Uşaklı 263c918088
feat: remove uid:<uid>:ignored:cids (#7099)
use cid:<cid>:ignorers instead
6 years ago
Barış Soner Uşaklı 26ccd8f626 fix: db info page
if there is no connection to db, connect first
6 years ago
Baris Usakli e65d40c937 dont return undefined from sortedSetsCard if keys is empty or falsy 6 years ago
Baris Usakli 420227faae parseFloat scores 6 years ago
Barış Soner Uşaklı 6330d51446 less derpy output 6 years ago
Barış Soner Uşaklı ed1bf9a16f add more info to advanced db page 6 years ago
Barış Soner Uşaklı 53ad2bbd6e
Categories optimize (#6999)
* WIP

* fix category page

* fix counts, and copyPrivileges

* fix lint

* more fixes,

* redis fix

* fix test

* fix  category test

* remove getParentsAndChildren
6 years ago
Barış Soner Uşaklı 0524fd9caa
closes #6996 (#6997)
* closes #6996

* fix lint
6 years ago
Baris Usakli 7b9a2caec3 closes #6994 6 years ago
Barış Soner Uşaklı 1c981e638b rename query name 6 years ago
Barış Soner Uşaklı ca1e3a758b #6986 6 years ago
Barış Soner Uşaklı 48690a01b9 dont need to call multi if keys.length is 0 6 years ago
Barış Soner Uşaklı 07f83c5078 fix redis tests 6 years ago
Barış Soner Uşaklı f8e219c323 handle empty set, add test 6 years ago
Barış Soner Uşaklı 75816deca7
add cache to redis (#6917)
* add cache to redis

move out cache module from mongo

* fix redis tests

* add callback noop

* fix typo

* del cache on field delete

* make redis/mongo caches separate
6 years ago
Barış Soner Uşaklı 0de60cfdca fix mongodb tests 6 years ago
Barış Soner Uşaklı 8ca36ad122 closes #6890 6 years ago
Barış Soner Uşaklı 7aa937a657 remove dupe code in mongodb sorted 6 years ago
Barış Soner Uşaklı c27be9db5a
Recent refactor (#6879)
* wip

* fix inf scroll

* remove duplicated code

* remove dupe code in /unread

* use topicList

* update tag page to use topicList

* fix tests

* combine ifs

* remove more dupe code

* disable timeout
6 years ago
Baris Usakli 51792da430 dont crash if there is nothing to write 6 years ago
Barış Soner Uşaklı fb6067a7ca only parse if field is requested 6 years ago
Baris Usakli 138154a71c move modules 6 years ago
Barış Soner Uşaklı b2b33ffa57 topics data refactor 6 years ago
Barış Soner Uşaklı 26d4e0852f use includes instead of indexOf
use _.uniq instead of filter&indexOf
6 years ago
Barış Soner Uşaklı 986511e718 try with higher timeout 6 years ago
Barış Soner Uşaklı 1d15dc089b revert isCluster 6 years ago
Barış Soner Uşaklı 0aebb9caf8 closes #6844 6 years ago
Barış Soner Uşaklı 154d7076d9 dont pass options 6 years ago
Barış Soner Uşaklı 9dbe68a9cb testing again 6 years ago
Barış Soner Uşaklı aa567a5f2d print out connString and opts 6 years ago
Barış Soner Uşaklı 2d8d234c57 test mongodb again 6 years ago
Barış Soner Uşaklı e615391b22 convert values to string before removing 6 years ago
Barış Soner Uşaklı 18d0deeccb
update deprecated methods (#6794) 7 years ago
Ben Lubar 11b655163e Only run the DDL for session storage on the primary instance. (#6788)
This fixes a restart loop caused by deadlocks in large clusters.
7 years ago
Barış Soner Uşaklı 9724ef083e
remove v from query 7 years ago
Barış Soner Uşaklı 786311fa64
use value 7 years ago
Barış Soner Uşaklı 5a69f979a8
update processSortedSet to use withScores 7 years ago
Ben Lubar 97a3079bc0 Optimize PostgreSQL session store.
- Sets session ID to be a constant-sized character field inline in the
  table instead of a variable-length character field compressed outside
  of the row.
- Changes the session data from "json" to "jsonb" which is faster and
  smaller on disk with the drawback of not maintaining key order and
  whitespace (which we don't care about at all)
- Adds a clustered index on the session expiration timestamp, which will
  make periodic session purging much faster.

(For an example, WTDWTF currently has 482355 sessions in the table, and
 because there is no index on the expiration timestamp, PostgreSQL must
 check all the rows in the table every time.)

This upgrade script can run concurrently with a live NodeBB instance,
but any action that touches sessions will wait until the upgrade script
is done.

Does not touch the database if the database driver is not set to
PostgreSQL or if there is a Redis database configured.
7 years ago
Barış Soner Uşaklı 0519f84734
Promisify modules (#6723)
* WIP promisify

* promisify psql

* ability to skip some keys

* dont promisify client object

* remove async

* clone entire module so it has all properties

* add shim for node 6

* ignore sessionStore as well

* ignore pool on psql
7 years ago
Ben Lubar 33228bb7fe PostgreSQL database driver (#5861)
* [test/database/list] Fix test list 4 being used in two different tests

* [database/postgres] PostgreSQL database driver

* [database/postgres] Make transactions work based on continuation scope.

* [database/postgres] Implement nested transactions

* eslint --fix

* Add database changes from earlier this week to the PostgreSQL driver.

* Fix typo

* Fix postgres.incrObjectFieldBy returning undefined instead of null when given NaN

* [database/postgres] Fix sortedSetsCard returning an array of strings.

* Update socket.io postgres adapter

* Fix PostgreSQL erroring when multiple updates are made to the same sorted set entry in a single operation.

Add a test case to catch this error.

* Fix lint errors.

* Only prune sessions on one instance in a cluster to avoid deadlocks.

They're caught and handled by the database server, but they spam the logs.

* Fix arguments.slice.
7 years ago
Baris Usakli 16a7728ab8 fix redis warnings 7 years ago
Barış Soner Uşaklı fe80186ddf add withscores 7 years ago
Barış Soner Uşaklı 8263f81d0c fix pubsub when only mongodb is used 7 years ago
Barış Soner Uşaklı 3b5d6951bb closes #6499 7 years ago
Barış Soner Uşaklı 77d47b31fb cleanly shutdown
wait for webserver to stop accepting connections
destroy current connections
wait for db connection to close
7 years ago
Baris Usakli 3c4b4f5263 don't call calback twice #6375 7 years ago
Barış Soner Uşaklı be5c237388 fix decr/incrObjectField if key in array doesn't exist create it 7 years ago
Barış Soner Uşaklı 6a9a73c86c changes to leaveAllGroups
Groups.destroy can take an array of groupnames
Groups.leave can take an array of groupnames
db.incrObjectField/decrObjectField can take an array of keys
db.sortedSetRemove can take an array of keys and values
db.setRemove can take an array of keys
7 years ago
Baris Usakli 38acedf5bb lint 7 years ago
Baris Usakli 959a742c63 add llen to dbal 7 years ago
Barış Soner Uşaklı bce640e514 closes #6340 7 years ago
Barış Soner Uşaklı 7a2d6fe22d Merge remote-tracking branch 'refs/remotes/origin/master' into develop 7 years ago
Barış Soner Uşaklı b084a596fa closes #6301 7 years ago
Barış Soner Uşaklı c0b7d4f5b5 fix redis export 7 years ago
Barış Soner Uşaklı 8a6345ded7 possible fix for pubsub 7 years ago
Ben Lubar e85aabbe74 Allow running as a cluster without Redis (#6233)
* [database/*] Allow databases other than Redis to provide pubsub for clustering if Redis is not present

* [pubsub] Delay messages sent before the database is ready until the database is ready.

* [pubsub] Restore old behavior of not using the database in non-clustered NodeBB instances.

See comment: https://github.com/NodeBB/NodeBB/pull/6233#issuecomment-357814968
7 years ago
Julian Lam 457194b333 Merge branch 'master' into develop 7 years ago
Barış Soner Uşaklı e092778ba6 closes #6200 7 years ago
Ben Lubar 5b1ed21634 [database/mongo] Improve speed of sortedSetRank (#6229)
* [database/mongo] Improve speed of sortedSetRank

* [database/mongo] Fix sortedSetRank to filter by _key
7 years ago
Barış Soner Uşaklı 5146f43b33 remove console.log 7 years ago
Barış Soner Uşaklı 87d40e9e78 fix increment to use data 7 years ago
Barış Soner Uşaklı cbaa977233 fix type on mongodb if key has expireAt 7 years ago
Barış Soner Uşaklı ea8cf6545c change db.set/get to use data field instead of value 7 years ago
Barış Soner Uşaklı 8e04e86e24 closes #5912 7 years ago
Baris Usakli 5ca3b535e7 closes #6105 7 years ago
Baris Usakli dbf815ed47 closes #6103 7 years ago
Baris Usakli 30ae0a5499 another process.nextTick 7 years ago
Baris Usakli e187a0def2 fix method name 7 years ago
Baris Usakli 12546cb9ad break up sync operation 7 years ago
Julian Lam 7e0d0e0eb5 Merge remote-tracking branch 'origin/develop' 7 years ago
Barış Soner Uşaklı 54a84bf130 cache fixes 7 years ago
Barış Soner Uşaklı 43f9c3224f dont crash sortedSetIncryBy on redis if no callback is supplied 7 years ago
Peter Jaszkowiak e609e497b3 Provide more error information to logs
Provides full stack instead of just message
7 years ago
Barış Soner Uşaklı a73c2628c4 cache improvements, stats 7 years ago
Barış Soner Uşaklı e7318a1cea Mongo cache (#6002)
* cache mongodb objects

remove user settings cache since it is cached at the object level

* removed indexOf from isMemberOfSets

* change has to get, change maxAge to 0
7 years ago
Barış Soner Uşaklı 7ad4420b6a closes #6000 7 years ago
Barış Soner Uşaklı 108f84fd19 add type to dbal 7 years ago
Barış Soner Uşaklı 62c81ca0d3 fix lint 8 years ago
Barış Soner Uşaklı 329bfeeb3b convert min/max to float 8 years ago
Barış Soner Uşaklı 923bd86662 Widgets refactor (#5817)
* widgets refactor

render widgets server side
widgets can use all the data the template can use

* fix tests
8 years ago
Baris Usakli e0de922ede Revert "widgets refactor"
This reverts commit 9e94edb5aa.
8 years ago
Baris Usakli 9e94edb5aa widgets refactor
render widgets server side
widgets can use all the data the template can use
8 years ago
Baris Usakli d469f53eba using async in batch
removed unused code in debug
8 years ago
Barış Soner Uşaklı 3d85992757 modify db.processSortedSet so it works with intervals 8 years ago
Baris Usakli 7fe239229d show 3 digits 8 years ago
Peter Jaszkowiak 3c2d4fe649 Switch from underscore to lodash 8 years ago
Barış Soner Uşaklı 890c2eff70 style changes 8 years ago
Barış Soner Uşaklı fe0a5aaf28 fix redis tests 8 years ago
Barış Soner Uşaklı fd0043f36a more tests 8 years ago
Barış Soner Uşaklı 95196ceedc closes #5692 8 years ago
Barış Soner Uşaklı 54a9e7eab9 #5674 8 years ago
Barış Soner Uşaklı 33f135c091 closes #5675 8 years ago
Julian Lam ad14b0d60b more fixes for #5660 8 years ago
Barış Soner Uşaklı 7135d25a4a closes #5654 8 years ago
Barış Soner Uşaklı 14f3907aa4 change the way group member posts are stored 8 years ago
barisusakli b91b41dfa3 show database values in gbs 8 years ago
barisusakli ba51a6c746 fix underscore 8 years ago
barisusakli b7b65c7d63 change style 8 years ago
barisusakli a2784738a4 fix lint 8 years ago
barisusakli 2029de4cc5 closes #5449 8 years ago
barisusakli 3d39a577f4 style change 8 years ago
barisusakli 38e16ca8f6 Merge branch 'master' into develop 8 years ago
barisusakli 65a65b19c0 tests for #5600 8 years ago
barisusakli c3ac7c31b1 Merge branch 'master' into develop 8 years ago
barisusakli 185b6e5c8e #5600 8 years ago
barisusakli e3205bcfad merge 8 years ago
barisusakli 9f8d80e3a4 closes #5600 8 years ago
barisusakli 92343eb2e1 fix mongo setObject if key/value is empty string 8 years ago
barisusakli 7ce6c1d0ad closes #5571 8 years ago
Peter Jaszkowiak 1ed571189c Make utils and translator easier to require
Move utils.walk to file.walk, backwards compatible
8 years ago
Peter Jaszkowiak 57696b22f4 Partial fix #5538 8 years ago
Barış Soner Uşaklı 6b0a907ede closes #5552 8 years ago
Julian Lam b6134b160e moved auto_reconnect out of socketOptions object for Mongo 8 years ago
barisusakli 5e0de4b8fb closes #5517 8 years ago
barisusakli 0c7dee05ef fix lint 8 years ago
barisusakli 92fa9fecb6 add mongodb connection options #3670
increase reconnect tries
add keepAlive setting
set autoReconnect to true, should be on by default
8 years ago
barisusakli 6aa8c8aa94 merge 8 years ago
barisusakli fa08498a9e closes #5480 8 years ago
Peter Jaszkowiak 4bb49a7171 ESlint no-unused-vars 8 years ago
Peter Jaszkowiak c18808fdeb ESlint no-useless-concat, no-mixed-spaces-and-tabs 8 years ago
Peter Jaszkowiak 09e868ce5f ESlint no-useless-escape, no-else-return 8 years ago
Peter Jaszkowiak 11cb3a9554 ESlint operator-assignment, block-scoped-var 8 years ago
Peter Jaszkowiak c4bdb72941 ESlint no-unneeded-ternary
and no-extend-native, no-sequences
8 years ago
Peter Jaszkowiak 896c8c7343 ESlint object-curly-spacing 8 years ago
Peter Jaszkowiak 1493afee2a ESlint indent 8 years ago
Peter Jaszkowiak 3b0dd2d1ef ESlint padded-blocks 8 years ago
Peter Jaszkowiak feb8405f95 ESlint eol-last 8 years ago
Peter Jaszkowiak a038c66549 ESlint quotes 8 years ago
Peter Jaszkowiak 604358ecc4 ESlint keyword-spacing, no-multi-spaces 8 years ago
Peter Jaszkowiak 2e031f3759 ESlint comma-spacing 8 years ago