* new verifier script with tests + telemetry
* dataDomainMismatch tests
* more tests for callbackStatus and plausibleInstalled
* create priv/verifier subfolder + fix Elixir CI
* bump CI cache version
* organize verifier tests
* Remove accidentally committed verifier
* Rework compilation: Make it a variant, always return new verifier code in tests
* Make priv/tracker/verifier/ exist
* Handle static checks with grace
* Fix paths
* Fix paths
* Add some tests
* Add one more test
* split up the JS
* proxyLikely + code structure refactor + unit tests
* fix telemetry fields
* move most telemetry to logs
* run verifier tests only on chromium
* detect wordpressPlugin and wordpressLikely
* detect GTM
* rename JS checks
* detect cookiebot
* include new fields in logs
* different logs for browserless request vs js failures
* detect manual extension
* detect unknown attrs + fix logging
* stick to Elixir checks for snippet detection
* fix codespell
* fix IO.inspect
* remove unnecessary fields from test mock
* cookiebot doc
* move test into verifier subfolder
* do not duplicate ts types
* comma -> semicolon in log
* test dynamically loaded snippet
* improve logging on Browserless error
---------
Co-authored-by: Karl-Aksel Puulmann <oxymaccy@gmail.com>
* Compile NPM modules into commonjs and esm modules
Note that since the module uses only vanilla es5 outside the export,
it's safe to just append to the minified output
* package.json, Typescript definition, generation for commonjs
* Scrap commonjs for the moment
Running into too big rabbit holes trying to get it off the ground
* Shared tests for npm and web
* Better way to identify npm module in fixture
* Simplify code
* Add NPM library specific tests
* Fix a flaky test
* Standardize a callsite
* npm_package standardize on
* Make legacy variant code more explicit
* Wrap code pre and post
* analyze-sizes.js should work with npm_package
* Port test changes from PR #5432
* Update outbound link feature test
* Await callInit
* Smaller npm module - mangle variables
* Clean up wrapping code
* chore: Bump tracker_script_version to 15
* Add debug messaging to expect
* Remove dead code
* Refactor, account for initialization delays in tests
* Remove needless features from compiler
As the compiler currently manages to compile everything in ~3 seconds,
the `features`/`compileIds` feature serves no more point. Scrapping it
* Re-inline readOutput
* Update tracker/test/plausible-npm.spec.js
Co-authored-by: Artur Pata <artur.pata@gmail.com>
* Remove redundant comment
---------
Co-authored-by: Artur Pata <artur.pata@gmail.com>
* Add CLI arguments to compile.js and logging
* Rename folder
* Extract compile code, es modules
* Add a progress bar
* Remove handlebars
* Update report-sizes
* Remove debug code
* inline
* More generous split
* Allow positional arguments for compiling
* Add watch option to compile
* Add compileFile logic
* Most tests run under playwright
* All tests runnable
* Update playwright, remove hack
Note that upgrading to latest failed due to a new test failure. This
might be due to a chrome update.
* Compile script on the fly for tests
* Minor refactor for compileAll
* es module for generate-variants.js
* Allow passing suffix to compilation script - this can be used to generate separate files for comparison
* Fix positionals
* Switch from 2 passes to 1 pass
Did some data analysis on this data:
- Compared to master, 1 pass increased brotli size by 0.7%, 2 passes 0.4%.
Given the change is insignificant enough, we can ignore it for now
The increase is likely due to order of operations in compilation and
some inlined functions getting lost.
* Move customEvents.js to plausible.js
* Clean up API
* Suffix default
* Rework variants.json, globals stored there
* Add more variants under test
* Distribute work across multiple worker threads
Compile time went on my machine from 60s -> 30s
* Fixup server
* Update canSkipCompile
* chore: Bump tracker_script_version to 7
* Update scripts
* Update node-version
* Experiment with adding a small delay to page
* Casing
* rename variable
* Update help text
* features -> compileIds, backport functionality from other branch
* Avoid compiling pageleave variant
Instead, it's treated as a no-op. Tracker still serves it.
Compile wall time went down from 86s -> 46s.
* Vendor, no longer compile p.js
This was a legacy script variant which we keep for backwards
compatibility but little else.
* chore: Bump tracker_script_version to 6
* move util function to util module
* move playwright config file to tracker dir root
* update Playwright + add gitignores
* Playwright: enable reuseExistingServer (non-CI env)
* store tracker src copies to avoid re-compilation in dev env
* test pageleave on simple navigation
* fix test util fn
* rename local_test npm script
* make test util able to expect multiple requests
* test pageleaves in SPAs
* test pageleave with manual URL
* test pageleave not sent in manual when pageview not triggered
* extend util fn to refute event requests
* test pageleaves not sent in excluded hash pages
* store hashes instead of file copies to detect /tracker/src changes
* drop async
* speed up test suite
This PR replaces geolix with locus to simplify self-hosted setup. locus can auto-update maxmind dbs which are recommended for self-hosters if they want city-level geolocation. locus is also a bit faster.
This PR also uses a test mmdb file from https://github.com/maxmind/MaxMind-DB for e2e geolocation tests without stubs.
* remove tracker files from git index
* generate tracker files on npm test
* generate tracker files for elixir tests/dev/CI
* update tracker/package-lock.json
* exclude npm run deploy from mix test + some docs
* Fix geolocation subdivision pattern matching
This commit fixes a bug where regions were not being saved. This was
caused because Geolix response was returning an additional
`:geolocation` map key. It also adds a test case for this.
Closes#2033
* Add geolocation database to .gitignore
* first commit with test and compile job
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* adding 'prepare' stage
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* updated ci script to include "test" compile phase
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* adding environment variables for connecting to postgresql
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* updated ci config for postgres
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* using non-alpine version of elixir
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* re-using the 'compile' artifacts and added explict env variables for testing
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* removing redundant deps fetching from common code
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* formatting using mix.format -- beware no-code changes!
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* added release config
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* adding consistent env variable for Database
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* more cleaning up of environment variables
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Adding releases config for enabling releases
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* cleaning up env configs
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Cleaned up config and prepared config for releases
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* updated CI script with new config for test
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Added Dockerfile for creating production docker image
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Adding "docker" build job yay!
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* using non-slim version of debian and installing webpack
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Adding overlays for migrations on releases
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* restricting the docker built to master branch only
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* typo fix
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* adding "Hosting.md" to explain hosting instructions
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* removed the default comments
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Added documentation related to env variables
* updated documentation and fixed typo
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* updated documentation
* Bumping up elixir version as `overlays` are only supported in latest version
read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Adding tarball assembly during release
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* updated HOSTING.md
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Added support for db migration
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* minor corrections
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* initializing admin user
Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you.
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Initial base domain update - phase#1
These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used.
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Using dedicated config variable `base_domain` instead
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* adding base_domain to releases config
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* removing the dedicated config "base_domain", relying on endpoint host
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Removed the usage of "Mix" in code!
It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Added support for SMTP via Bamboo Smtp Adapter
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Capturing SMTP errors via Sentry
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Minor updates
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Adding junit formatter -- useful for generating test reports
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* adding documentation for default user
* Resolve "Gitlab Adoption: Add supported services in "Security & Compliance""
* bumping up the debian version to fix issues
fixing some vulnerabilities identified by the scanning tools
* More updates for self-hosting
Changes in most of the places to suit self-hosting. Although, there are some which have been left-off.
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* quick-dirty-fix!
* bumping up the db connect timeout
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* bumping up the db connect timeout
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* bumping up the db connect timeout
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* bumping up timeout - skipping MRs :-/
* removing restrictions on watching for changes
this stuff isn't working
* Update HOSTING.md
* renamed the module name
* reverting formatting-whitespace changes
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* reverting the name to release
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* adding docker-compose.yml and related instructions
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* using `plausible_url` instead of assuming `https`
this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* WIP: merging changes from upstream
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* wip: more changes
* Pushing in changes from upstream
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* changes to ci for testing
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* cleaning up and finishing clickhouse integration
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* updating readme with hosting details
* removing deleted files from upstream
* minor config adjustments
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* formatting changes
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* changing the connection strategy for clickhouse during release
since clickhouse integration doesn't have an ecto support, we need to prepare the db _before_ the clickhouse migration. One workaround is to connect to a default db on init and then create a db
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* formatting
* cleanup and added separated migration to setup
* Big improvements to selfhosting
- added ability for disabling
- authentication completely
- registration
- landing page
- formatting cleanups
* Big improvements to selfhosting
- added ability for disabling
- authentication completely
- registration
- landing page
- formatting cleanups
* changing smtp auth to optional
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* removed stale templates and permanently removed landing page
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* removed stale templates and permanently removed landing page
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* removed stale templates and permanently removed landing page
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* WIP
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* fixes form upstream merge
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* added disabling subscription for selfhosted version
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* updated doc
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* formatting
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Remove reference to file that doesn't exist
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* do not show direct traffic if there's no data
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* addressing PR comments
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* formatting
Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
* Use oban for work
* Remove mix task for email reports
* Convert remaining tasks to Oban workers
* Add crontab for scheduled jobs
* Only enable cron if configured to do so
Adam Rutkowski
RobertJoonas
Karl-Aksel Puulmann
Artur Pata
ruslandoga
Uku Taht
Vinicius Brasil
Manu S Ajith
Oliver Kriska
Johan Fagerberg
Chandra Tungathurthi
Rimenes Ribeiro