* add module name to service_error when check times out
Otherwise, it can sometimes remain unclear in the diagnostics, whether
it was InstallationV2 or InstallationV2CacheBust that timed out.
* Remove duplicate timeout logic
The current production logs show two types of verification timeouts:
* service_error: "Unhandled Browserless response status: 408" (vast
majority of cases)
* service_error: :timeout (only a few cases)
The latter happens when we hit the Req receive_timeout
(endpoint_timeout + 2s). I've seen Browserless not respect the timeout
param from time to time, so it's better to keep the timeout logic
"in-house" only.
* make service_error into a map with code and extra
* interpret temporary service errors
...but still consider them "unhandled" for telemetry, also notifying Sentry
and logging the warning.
* separate sentry messages (verification)
* make Verification.ChecksTest more DRY
* organize tests into describe blocks
* test verification telemetry and logging
* fix codespell
* get rid of legacy verification
* rename Checks.InstallationV2 -> Checks.VerifyInstallation
* delete Live.Installation and rename Live.InstallationV2 -> Live.Installation
* rename installationv2 (live) files as well
* delete old change-domain routes
Also rename current liveview modules and routes, removing the v2 suffix
* rename domain_change_v2 files, removing v2 suffix
* remove legacy JS verifier code
Also fix dockerignore and elixir.yml referencing a wrong priv path
* rename verification_v2_test -> verification_test
* remove v2 prefix from logs and sentry messages
* clean up duplicate external_sites_controller_test.exs tests
* remove flag
* fix typespec
* pass timeout as query param to Browserless too
* Fixup external sites controller test module (#5826)
* fix test description
---------
Co-authored-by: Artur Pata <artur.pata@gmail.com>
* Start passing official installs that are using legacy .compat installation
* Pass the first domain for verifying legacy compat installs
* Fix variable access issue
* Fix non-existent function call
* Comment why split domainString
* Add logic to find Plausible script tag
* Clarify where cookies fixtures come from
* Add case for snippet not found
* Fix issue with installation type not propagating to diagnostics
* Better struct definition
* Refactor verification tests to be more comprehensive
* Fix compile error related to Test.Support.DNS
* Fix DNS imports for CE
* Refactor when tracker is in HTML is detected
* Show Plausible not detected errors with installation type specific recommendations
* Update diagnostics and diagnostics tests for verification
* Format
* Stop calling statuses between 200..299 as errors
* Remove function that populates diagnostics randomly
* Accept cookies (WIP)
* Make resolving Plausible function call a priority
* Update tracker script version
* Remove cookieBannerLikely, replace with debug-only cookiesConsentResult
* Use compact rules
* Make verifier tests less flaky
* Add fixtures for accepting cookies
* Limit CMPs
* Fix dep position
* Improve "back to settings" button
* Dark mode support for change domain
* Purge CDN cache on domain change
* Allow npm installation_type
* Detect npm installation type in detector
* Support npm installation type in onboarding
* Show warning in change domain flow for npm
* Make CE tests happy
* Cleanup
* npm_likely -> npm
* Cleanup
* Add fast failing dns check to verification
* Convert Detection to a checks pipeline
* Convert detection to checks pipeline
* Unify browserless checks, set retry policy, timeouts
* Fix spelling
* Update change domain v2
* Fix issue with handling errors with detection
* Include timeoutMs in detector function args
* Allow saving npm installation type (#5639)
* small code style/comment improvements
---------
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* Handle canceled execution context when verifying v2 installs
* Show diagnostics even on verification success for superadmins
* Add check for scriptv2 user to support branching off to v2 site verification better
* fix comment on localhost dogfood tracking
* improve detector script and integrate into Elixir
* wait for window.plausible.l instead of window.plausible
* do not touch source files during compilation
* stop referencing compiler hint module attr
* new verifier script with tests + telemetry
* dataDomainMismatch tests
* more tests for callbackStatus and plausibleInstalled
* create priv/verifier subfolder + fix Elixir CI
* bump CI cache version
* organize verifier tests
* Remove accidentally committed verifier
* Rework compilation: Make it a variant, always return new verifier code in tests
* Make priv/tracker/verifier/ exist
* Handle static checks with grace
* Fix paths
* Fix paths
* Add some tests
* Add one more test
* split up the JS
* proxyLikely + code structure refactor + unit tests
* fix telemetry fields
* move most telemetry to logs
* run verifier tests only on chromium
* detect wordpressPlugin and wordpressLikely
* detect GTM
* rename JS checks
* detect cookiebot
* include new fields in logs
* different logs for browserless request vs js failures
* detect manual extension
* detect unknown attrs + fix logging
* stick to Elixir checks for snippet detection
* fix codespell
* fix IO.inspect
* remove unnecessary fields from test mock
* cookiebot doc
* move test into verifier subfolder
* do not duplicate ts types
* comma -> semicolon in log
* test dynamically loaded snippet
* improve logging on Browserless error
---------
Co-authored-by: Karl-Aksel Puulmann <oxymaccy@gmail.com>
* Sync pathname to event.props.path for special path-based goals
* Stop adding event.props.path for 'Form: Submission' events
* Update tracker script version
* Fix test expectations
* Fix format
* Simplify maybe_put_props_path with 'with'
* Add specs, fix factory
* Update tracker changelog
* Update EE/CE changelog
* Remove business logic from factory
* Refactor event.props.path sync result to be validated with the rest of the custom props
* Clarify doctests and update function name
* Do not send duplicate tagged form events
* Update changelog
* chore: Bump tracker_script_version to 22
* Share HTML fixture
---------
Co-authored-by: apata <apata@users.noreply.github.com>
* Make delays conditional on COMPILE_COMPAT, otherwise rely on keepalive fetch
* Add tests
* Fix webkit related test file-download feature tests issues
* Remove ternaries for scriptConfig in tests
* Make sure .compat scripts don't contain unreachable code blocks
* Test legacy .compat extension in a separate block
* Tests for tagged events
* Remove extraneous tests, add needed tests
* Add tests for v2 features overlap and outbound link clicks
* Add changelog, add current behaviour test for outbound links
* Remove mockRequest, unify API
* Fix coverage of legacy compat navigation, add back interactive event test
* chore: Bump tracker_script_version to 21
* Update specs
* Fix issues with _ variable
* Fix args
* Types and formatting cleanup
---------
Co-authored-by: apata <apata@users.noreply.github.com>
* Revert "Log warning when m/meta parameter used in ingestion (#5478)"
This reverts commit d4bec6d9e3.
* Support `meta` argument only on legacy scripts + test
* Changelog
* Bump version
* Commit as Plausible Bot
This way commits by it can be excluded from protections
* ScriptV2: Add support for `config.transformRequest`
transformRequest allows users to either manipulate or ignore requests to plausible
with a minimal footprint.
Some use-cases we're aware of and are planning for:
1. Ignoring requests similar to previous exclusion rules in WordPress
2. Normalizing urls for requests
* Allow passing `options.url` for overriding url when calling track
Previous naming `u` was unintuitive, but is kept around (untyped) for backwards
compatibility reasons
* chore: Bump tracker_script_version to 17
* Changelog
* Update types
* Docs
* Add test showing interaction with engagement events
* README.md
* Compile NPM modules into commonjs and esm modules
Note that since the module uses only vanilla es5 outside the export,
it's safe to just append to the minified output
* package.json, Typescript definition, generation for commonjs
* Scrap commonjs for the moment
Running into too big rabbit holes trying to get it off the ground
* Shared tests for npm and web
* Better way to identify npm module in fixture
* Simplify code
* Add NPM library specific tests
* Fix a flaky test
* Standardize a callsite
* npm_package standardize on
* Make legacy variant code more explicit
* Wrap code pre and post
* analyze-sizes.js should work with npm_package
* Port test changes from PR #5432
* Update outbound link feature test
* Await callInit
* Smaller npm module - mangle variables
* Clean up wrapping code
* chore: Bump tracker_script_version to 15
* Add debug messaging to expect
* Remove dead code
* Refactor, account for initialization delays in tests
* Remove needless features from compiler
As the compiler currently manages to compile everything in ~3 seconds,
the `features`/`compileIds` feature serves no more point. Scrapping it
* Re-inline readOutput
* Update tracker/test/plausible-npm.spec.js
Co-authored-by: Artur Pata <artur.pata@gmail.com>
* Remove redundant comment
---------
Co-authored-by: Artur Pata <artur.pata@gmail.com>
* Track form submissions
* chore: Bump tracker_script_version to 11
* Check that script loadedin tests
* Track form submissions
* Fix format
* Fix missing form submit handler
* Make id casing consistent
* Automatically sync compiled script snippet in dynamic test pages
* Enforce format manually
* Skip one test on Firefox, explain reason for stub
* Revert "Automatically sync compiled script snippet in dynamic test pages"
This reverts commit 7622b23f31.
---------
Co-authored-by: apata <apata@users.noreply.github.com>
* WIP: init() for plausible.js
* Reformat whitespace
* Handle double-initialization gracefully
* Script customProperties option
* Remove pageview-props from tracker script interpolation - its now an advanced config
* Add compile option for compiling the web snippet. This can then be copied to e.g. onboarding
* Switch minification from uglify-js to swc
* Use 4 passes for optimal output
* Remove use strict to remove it from output. Uglifyjs already removed this
* Minor optimization: Remove nulls from output via runningEngagementStart
* Minor optimization: Reduce onIgnoredEvent
* config_json -> config_js
* Dead attribute
* Remove guard for engagements
* More protective customProperties
* Object.assign
* chore: Bump tracker_script_version to 10
* No need for default in Object.assign
* Remove dead guard
* Add CLI arguments to compile.js and logging
* Rename folder
* Extract compile code, es modules
* Add a progress bar
* Remove handlebars
* Update report-sizes
* Remove debug code
* inline
* More generous split
* Allow positional arguments for compiling
* Add watch option to compile
* Add compileFile logic
* Most tests run under playwright
* All tests runnable
* Update playwright, remove hack
Note that upgrading to latest failed due to a new test failure. This
might be due to a chrome update.
* Compile script on the fly for tests
* Minor refactor for compileAll
* es module for generate-variants.js
* Allow passing suffix to compilation script - this can be used to generate separate files for comparison
* Fix positionals
* Switch from 2 passes to 1 pass
Did some data analysis on this data:
- Compared to master, 1 pass increased brotli size by 0.7%, 2 passes 0.4%.
Given the change is insignificant enough, we can ignore it for now
The increase is likely due to order of operations in compilation and
some inlined functions getting lost.
* Move customEvents.js to plausible.js
* Clean up API
* Suffix default
* Rework variants.json, globals stored there
* Add more variants under test
* Distribute work across multiple worker threads
Compile time went on my machine from 60s -> 30s
* Fixup server
* Update canSkipCompile
* chore: Bump tracker_script_version to 7
* Update scripts
* Update node-version
* Experiment with adding a small delay to page
* Casing
* rename variable
* Update help text
* features -> compileIds, backport functionality from other branch
* Avoid compiling pageleave variant
Instead, it's treated as a no-op. Tracker still serves it.
Compile wall time went down from 86s -> 46s.
* Vendor, no longer compile p.js
This was a legacy script variant which we keep for backwards
compatibility but little else.
* chore: Bump tracker_script_version to 6
* WIP
* tracker_script_version over reportedVersion
Will make it easier to grep for this thing
* CHANGELOG.md
* Only add tracker/package.json in workflow commit
* chore: Bump tracker_script_version to 1
* Show tracker_script_version is reported in tracker tests
* Track `engagement_time` while accounting for time page is blurred (#5206)
Consider this scenario:
1. User opens page /blog
2. 3 seconds later, alt-tabs to another window
3. minute later, alt-tabs to the page and closes it
Previously, this would be reported as 1m03s engagement_time. Now this
would be reported as 3s engagement_time.
* remove redundant engagement cooldown (#5199)
---------
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
* Track engagement time in plausible.js
* Test page going between background and foreground without sending
* Rename some variables
* Add comment on when we send information or not
* Extract a function
* Test SPA navigation
* Raise threshold
* Update tracker/src/plausible.js
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
---------
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
`fetch` allows us to use the keepalive flag, which more reliably
delivers data to us. It's also required for reliable scroll depth
capturing down the line.
The only major browser not to support fetch is Internet Explorer.
Clients who care about IE traffic can use the `.compat` variant of our
script (documented in
https://plausible.io/docs/script-extensions#scriptcompatjs). As of this
commit, IE makes up less than ~0.003% of incoming traffic to plausible.
* Separate `engagement` event triggered on visibilitychange
* Set up shared tests for engagement requests and pageleaves
* Store scroll_depth on engagement events
* Move test to be shared
* Update cache_store
RobertJoonas
Artur Pata
Karl-Aksel Puulmann
Uku Taht