Commit Graph

349 Commits

Author SHA1 Message Date
Adam Rutkowski aed90f7ffc
Make CI vigilant about uncaptured logs during test runs (#5676)
* Let's see

* test error

* Revert test changes

* Bump

* schedulers

* tmp

* Bump timex

* ignore libcluster warning

* fixup

* fix typo

* Set shell: bash
2025-12-15 13:28:04 +00:00
Adam Rutkowski 111a8b9462
Enforce max limit for goals per site (#5917)
* Limit preloading goals

* Enforce max limit for goals per site

* typo

* credo

* Remove logger call

* Integrate #5916

* Add a test

* Add test

* Unignore opts
2025-11-27 10:19:13 +00:00
RobertJoonas a83b4f3583
Clean up legacy verification code and script v2 flag (#5824)
* add module name to service_error when check times out

Otherwise, it can sometimes remain unclear in the diagnostics, whether
it was InstallationV2 or InstallationV2CacheBust that timed out.

* Remove duplicate timeout logic

The current production logs show two types of verification timeouts:

* service_error: "Unhandled Browserless response status: 408" (vast
  majority of cases)
* service_error: :timeout (only a few cases)

The latter happens when we hit the Req receive_timeout
(endpoint_timeout + 2s). I've seen Browserless not respect the timeout
param from time to time, so it's better to keep the timeout logic
"in-house" only.

* make service_error into a map with code and extra

* interpret temporary service errors

...but still consider them "unhandled" for telemetry, also notifying Sentry
and logging the warning.

* separate sentry messages (verification)

* make Verification.ChecksTest more DRY

* organize tests into describe blocks

* test verification telemetry and logging

* fix codespell

* get rid of legacy verification

* rename Checks.InstallationV2 -> Checks.VerifyInstallation

* delete Live.Installation and rename Live.InstallationV2 -> Live.Installation

* rename installationv2 (live) files as well

* delete old change-domain routes

Also rename current liveview modules and routes, removing the v2 suffix

* rename domain_change_v2 files, removing v2 suffix

* remove legacy JS verifier code

Also fix dockerignore and elixir.yml referencing a wrong priv path

* rename verification_v2_test -> verification_test

* remove v2 prefix from logs and sentry messages

* clean up duplicate external_sites_controller_test.exs tests

* remove flag

* fix typespec

* pass timeout as query param to Browserless too

* Fixup external sites controller test module (#5826)

* fix test description

---------

Co-authored-by: Artur Pata <artur.pata@gmail.com>
2025-10-27 09:39:41 +00:00
Artur Pata f01ce1b1b6
Script v2: Identify detection attempt (#5818)
* Identify detection attempt

* Test Detection.Checks module

* Fix typo
2025-10-22 10:16:54 +00:00
Adrian Gruntkowski c18d871d52
Fix parsing of percentage setting for persistor from env in runtime config (#5772) 2025-10-01 13:27:09 +00:00
Uku Taht 6cdf8016e7
Tailwind v4 (#5756)
* WIP

* Add tailwind migration script

* Change deprecated classes via migration script

* Manual fixes

* Restore color configuration with CSS variables

* Convert the rest of the config

* Fix dropdown ring opacity

* Remove migration script

* Fix stylelint errors

* NPM run format

* Fix shadow and rounded variant usage

* Add custom CSS to components layer

* Fix dark: and hover: conflict in settings_tab

* Fix funnel form background opacity

* Fix final bg-opacity

* Bring back cursor-pointer for buttons by default

* npm run format
2025-09-29 15:21:15 +00:00
Adrian Gruntkowski d170d9cc9c
Implement directing percentage of traffic to a given persistor backend (#5700)
* Implement directing percentage of traffic to a given persistor backend

* Fix percentage check

* Add tests for percentage driven backend switch

* Fix typo
2025-09-23 12:42:58 +00:00
Adrian Gruntkowski 4754e2a3e8
Remove SSO_ENABLED env flag and replace it with ee? checks where needed (#5728)
* Remove `SSO_ENABLED` env flag and replace it with `ee?` checks where needed

* Fix name of a test module to avoid clash

* Remove unnecessary `ee?()` check from condition in `extra/` code
2025-09-16 14:18:59 +00:00
Artur Pata f7acb068d4
Script v2: Change priority of CSP related error for verifying manual installations (#5707)
* Add logic to find Plausible script tag

* Clarify where cookies fixtures come from

* Add case for snippet not found

* Fix issue with installation type not propagating to diagnostics

* Better struct definition

* Refactor verification tests to be more comprehensive

* Fix compile error related to Test.Support.DNS

* Fix DNS imports for CE

* Refactor when tracker is in HTML is detected
2025-09-11 06:59:03 +00:00
Adrian Gruntkowski 3fbea514a6
Use embedded persistor implementation in local dev env (#5688) 2025-09-01 11:34:25 +00:00
Adrian Gruntkowski 40ff36a4fb
Implement dedicated persistence service (#5653)
* Implement a very crude HTTP relay to persistor service

* Temporarily disable local session and event persistence

* Temporarily disable Promex in dev

* Setup dedicated Finch pool for persistor

* Temporarily adjust load script parameters

* Fiddle with pool opts

* Only log errors

* Bump rate

* Put persistor configuration in runtime config

* Move persistor client code to a separate module

* Move event and session persistence behind a switchable adapter

* Add scaffolding of relaying persistor

* Instrument relayed persistence

* Adjust persistor API to accept full ingest event

* Fix persistor URL in config defaults

* Switch local dev env of embedded_with_relay backend

* Revert "Temporarily disable Promex in dev"

This reverts commit d9c9e9075d130cc5177a2a7559b98a2026077327.

* Lower default `PERSISTOR_COUNT`

* Refactor payload encoding and decoding slightly

* Test and slightly improve persistor logic
2025-09-01 09:13:41 +00:00
Artur Pata 276f95cda2
Script v2: Make detection take less time (#5635)
* Add fast failing dns check to verification

* Convert Detection to a checks pipeline

* Convert detection to checks pipeline

* Unify browserless checks, set retry policy, timeouts

* Fix spelling

* Update change domain v2

* Fix issue with handling errors with detection

* Include timeoutMs in detector function args

* Allow saving npm installation type (#5639)

* small code style/comment improvements

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
2025-08-19 10:16:27 +00:00
Uku Taht 00f24bfcd6
Use new Detection module in installation screen (#5604)
* Use new Detection module in installationv2

* Update tests

* Simplify mount function
2025-08-04 13:42:51 +00:00
RobertJoonas 97dcc3fe7c
Refactor Verification module structure (#5570)
* detector.js

* refactor: organize modules better

* Renaming (Elixir + JS)

* lib/plausible/verification -> lib/plausible/installation_support
* test/plausible/verification -> test/plausible/installation_support
* priv/tracker/verifier -> priv/tracker/installation_support
* tracker/verifier -> tracker/installation_support
* tracker/test/verifier -> tracker/test/installation-support

* rename remaining test modules

* add documentation

* dialyzer: remove module refs that do not exist yet

* Fix CI

* fix tracker CI

* fix tracker CI for good
2025-07-15 10:50:34 +00:00
Adam Rutkowski 69b180dc21
Bugfix: enable nameservers fallback when no DNS configured explicitly for SSO (#5530)
* Make sure no empty nameservers list is passed to `:inet_res.lookup`

* Never use empty list of nameservers when reading env vars
2025-06-25 14:19:35 +00:00
Adam Rutkowski e56baeb272
Persist login type preference (SSO/standard) (#5520)
* First pass: store login preference

* Only set login preference if SSO is used

* Change mock DNS to use port 5354 and `domain_id` for parameter

* Make login forms use flash message for error passing

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2025-06-23 10:28:27 +00:00
Karl-Aksel Puulmann 6ade93bf86
ScriptV2: Self hosted tracker script cache (#5502)
* Allow caching tracker script on CE

Open questions with this approach:
- `ingestion_url`: Using `PlausibleWeb.Endpoint.url()` requires that endpoint has started, but we
  want to pre-warm the cache _before_ the endpoint starts. To work around this, a different approach
  is used to get the right url.
- caching: Other caches currently cache database models, this caches a string. Will this cause issues?

* Slightly better workaround

* Lazier timers
2025-06-20 06:59:18 +00:00
Adam Rutkowski 37e718db40
End-to-end sso domain verification (#5513)
* wip

* Fix up typespcs

* Extract domain status enum to module macros

* Cancel verification jobs when integration is removed

* Revisit verification interface

* Supply local DNS server for domain ownership testing

* Rename a test

* 👾

* Use identifier when submitting domain verification

* Disallow re-verification of already verified domains
2025-06-18 09:52:48 +00:00
Adam Rutkowski 70902bbc5a
SSO: phrasing; settle on domain verification, not validation (#5508)
* SSO domains: s/validation/verification

* Fix careless search&replaces
2025-06-16 18:44:57 +00:00
Adam Rutkowski 1c182d6980
Set a long salts refresh interval on Mix.env == :test (#5510) 2025-06-16 14:50:01 +00:00
Adam Rutkowski b86e4eea7e
SSO: implement domain validation background service (#5505)
* SSO: implement domain validation background service

* Put tests behind `ee`

* Fix up typespecs
2025-06-16 11:56:59 +00:00
Adrian Gruntkowski 153702c20f
Implement SSO setup UI (#5473)
* Extend core APIs for better insight into SSO state during setup

* Adjust `toggle_switch` compoenent to accept global attributes

* Improve textarea variant of input component

* Make first very crude pass on SSO setup UI

* Display "Single Sign-On" settings option only when enabled

* Extract SP identity ID function to SAMLConfig as a public one

* Move SAML controller logic behind an adapter

* Don't apply CSRF protection to SAML consume endpoint

* Make copy adjustments

* Change SSO sidebar icon

* Fix `fake_domain_verify` handling before integration is set up

* Tweak configuration markup

* Sanitize PEM input

* Tweak SSO settings sections + prevent domain deletion if applicable

* Minor tweaks to SSO settings tiles

* Bootstrap basic test suite

* Ensure SSO settings are guarded by env var

* Make sure to retry fake domain validation on integration-less cycle

* Trim idp entity id input

* Dedup assertions

* Fix domain identifier interpolation

* Tidy up integration fetching

* Expand tests for SSOManagement LV with fixes

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2025-06-11 10:52:20 +00:00
hq1 efc55e323d
Remove kaffy (#5423)
* Deal with `FeatruesList` proxy

* Remove kaffy: first pass

* Remove admin controller

* Remove kaffy: last batch

* unlock dependency

* Remove kaffy links

* !fixup
2025-06-10 06:24:47 +00:00
Adrian Gruntkowski bca78169bb
Implement SSO login against fake SAML endpoints (#5434)
* Add SSO user as a team member on provisioning

* Implement fake SSO actions and basic login form

* Handle team member limit error and adjust login redirect in `UserAuth`

* Always switch to related SSO team on SSO user login

* Ensure `timeout_at` is set when creating new standard user session

* Add env var flag for gating SSO

* Hide SSO login link when SSO disabled

* Hide SSO routes when SSO disabled

* Implement nonce-based allowance for js in SSO content security policy

* Test controller actions

* Add more tests for UserAuth

* Add sync tests for `GateSSO` plug and env var flag in disabled state

* Add test for SSO owner logging in via standard login
2025-06-02 12:13:56 +00:00
Karl-Aksel Puulmann 1ba438fe8d
Make scheduled jobs work in development (#5436)
As part of https://github.com/plausible/analytics/pull/5435, I could not
get scheduled jobs working locally. After removing `plugins: false`
(defaults to empty list) it works, got an idea from reading  unrelated
bug reports.
2025-05-28 10:08:44 +00:00
Karl-Aksel Puulmann 6a06020ba7
ScriptV2: BunnyCDN cache purging (#5435)
* Purge bunny cdn cache when tracker script config is updated

* Only allow one purge task scheduled per site

Tested by adding logging and toggling a checkbox a few times. Only one job got executed.

* Typo
2025-05-28 10:08:33 +00:00
RobertJoonas 2dd144bf85
Starter tier: Upgrade page remodelling (#5394)
* add a new (feature flagged) upgrade page offering v5 plans

* include starter tier plans in available_plans_for + use dev prices in test

* upgrade page remodelling with starter tier

* mobile optimizations

* optimize for darkmode

* add embedded dashboards as a growth benefit

* do not hide header on LegacyChoosePlan

* consistent v5 plan feature order

* slight grandfathering notice adjustment

* display monthly price too on yearly plans

* default to v5 plans unlesss legacy? is true

* refactor: suggest volume not plan for emails

* align back link with page title

* render grandfathering notice for growth v4 too
2025-05-20 13:22:12 +00:00
ruslandoga 97449613e1
Support `sslmode` in `DATABASE_URL` in CE (#5280)
* sslmode

* Update config/runtime.exs

---------

Co-authored-by: Cenk Kücük <cenk@plausible.io>
2025-05-20 13:21:48 +00:00
hq1 8f4b63083e
Loadtest p2 (#5380)
* Bring back previous dev config

* Dedicated MIX_ENV for load testing + change k6 script

* Remove unused env var

* Remove test flag

* Add .env.load

* Fixup
2025-05-06 10:58:06 +00:00
hq1 aa4a8339cb
Ingest throughput fixes (#5378)
* Update

* Update

* Naive safety valve in front of RL

* Revert "Naive safety valve in front of RL"

This reverts commit 3bb553ec2e.

* rate limit with atomics

* update test

* Reapply "Add +Mdai max emulator flag (#5373)" (#5374)

This reverts commit b28ca2ffee.

* Update load script

* Update LOADTEST mode

* Revert "Stop aggregating buffered ingest counters (#5372)"

This reverts commit 2c41dcd4c1.

* update

* Fix cache hit/miss metric tags

---------

Co-authored-by: ruslandoga <ruslandoga+gh@icloud.com>
2025-05-05 14:00:37 +00:00
RobertJoonas cb465511bb
Streamline creating/updating/deleting subscriptions in development (#5328)
* streamline dev subscriptions

* spawn -> Task

* fix button alignment

* do not define routes in test env

* remove explicit mentions of dev env when deciding whether to sandbox paddle

* decide checkout behaviour at compile time

* Timex -> Date

* hum

* make it work for enterprise plans too

* allow convenient subscription status change

* fix ci warnings

* remove redundant commented out code

* fix crash + review suggestion

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2025-04-16 13:43:17 +00:00
ruslandoga 9bfb1992d9
Sessions transfer (#5229)
* sessions transfer

* took on ignore

* add slow test

* more tests

* allow BIG messages

* update tests

* continue

* continue

* cleanup

* fewer changes

* add config tests

* todo

* use less app env

* oops

* fixes

* cleanup

* update tests

* remove useless tests

* more buckets

* Update lib/plausible/session/transfer.ex

* Update transfer.ex

* Update lib/plausible/session/transfer.ex

* avoid calling into ConCache directly

* cleanup, add docs

* Update lib/plausible/session/transfer.ex

* fewer options

* there is no loop

* force deploy

* force deploy again

* individual puts

* fewer changes in cache/adapter.ex

* oops

* use existing atom names

* Cosmetic changes

* Bring back slow tag

---------

Co-authored-by: hq1 <hq@mtod.org>
2025-04-16 09:56:39 +00:00
Karl-Aksel Puulmann 42cea1d889
time-on-page: `site.legacy_time_on_page` cronjob (#5215)
* site.legacy_time_on_page_cutoff migration

* Update migration logic

* Cronjob for setting legacy_time_on_page_cutoff

* Test site creation default legacy_time_on_page_cutoff

* Use site.legacy_time_on_page_cutoff

Leaving the query param logic for now for testing purposes

* tweak cron

* Work around defaults in tests

* Fixes

* Make cutoff date in job configurable - useful for backfilling
2025-03-31 08:54:49 +00:00
ruslandoga 8926f6d6c1
peep (#5130)
* peep

* adapt peep to prom_ex buckets

* handle overmax
2025-02-28 13:15:05 +00:00
hq1 9e5be7103a
Revert "peep (#5126)" (#5128)
This reverts commit 90d0581c99.
2025-02-28 08:00:48 +00:00
ruslandoga 90d0581c99
peep (#5126) 2025-02-28 06:46:16 +00:00
ruslandoga 66c57daa70
use 100 sessions partitions (#5121) 2025-02-26 17:45:36 +00:00
hq1 ccdf50b734
Balance sessions across worker queues instead of explicit locks (#5113) 2025-02-26 08:03:52 +00:00
hq1 78689c89a6
Bump Repo queue params (#5109) 2025-02-24 16:23:23 +00:00
Adrian Gruntkowski bf010a1537
Implement support for multiple team owners and multiple teams per user (#5008)
* Add tests for `Teams.get_or_create/1` and `Teams.get_by_owner/1`

* Start populating `current_team` in assigns fetching value from session

* Clean up team passing in invitation services

* Make site transfer service handle multi-team scenario

* Handle multi-team and permission transfer errors on controller level

* Handle multi-teams in site creation on service and controller level

* Drop validation limiting full membership to a single team

* Make user deletion account for public team ownership

* Adjust feature availability checks for Stats API key

* Use current_team when determining limits on site transfer invitation

* Adjust trial upgrade email submission to account for multiple owners

* Remove unnecessary `Teams.load_for_site/1`

* Spike renaming `owner` and `ownership` relationships to plural versions

* Make HelpScout integration handle owner of multiple teams gracefully

* Add FIXME note

* Resolve paddle callback issue by always provisioning a new team when none passed

* Set `current_team` as `my_team` only when user is an owner

* Implement basics of Teams CRM

* Extend Teams CRM

* Further adjust User and Site CRM and refine Team CRM

* Convert Enterprise Plan CRM to refer to team directly and not via user

* Remove unused virtual fields from User schema

* Add note to HelpScout integration

* Allow listing multiple owners under Site Settings / People

* Remove unused User schema relations

* Fix current team fetch in auth plug and context

* Implement basic team switcher

* Ensure (site) editor role is properly handled in site actions auth

* Don't set `site_limit_exceeded` error marker on `permission_denied` error

* Link from HS integration to Team CRM instead of User CRM when available

* Ensure consistent ordering of preloaded owners

* Add `with_subscription` preload for optimisitation

* Add ability to search sites by team identifier

* Add ability to pick team when transferring ownership directly

* Fix failing HelpScout tests

* Scope by team when listing sites in dashboard and via API (optional)

* Add ability to search by team identifier in plans CRM lookup widget

* Add subscription plan, status and grace period to team status info

* Expose teams list in user CRM edit form and fix team details CRM view

* Fix Team Switcher styling

* Reorganise header nav menu

* Avoid additional queries when authenticating user

* Hide the pay/site transfer message on lock screen when teams FF is on

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2025-02-19 09:33:25 +00:00
hq1 aa64d7ba4c
Update oban + sentry (#5088)
* Update oban + sentry

* Fixup

* dialyzer

* Add Reindexer plugin daily at 1am
2025-02-18 15:27:31 +00:00
Adrian Gruntkowski a0c13383e7
Implement support for cache partitioning for sessions (#5073)
* Implement very rudimentary support for cache partitioning for sessions

* Convenience for starting partitioned caches

* Test basic partitioning expectations

* Include put_many in test

* Use div/2

* Remove unused alias

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2025-02-17 08:38:34 +00:00
Uku Taht 6c30f62d5d
Upgrade deps, add storybook (#4947)
* Add `<.dropdown_item>`

* Make the ellipsis menu functional again

* Upgrade deps so that storybook can be added

* Add storybook and dropdown story

* Remove lingering warnings/errors

* Add color mode to storybook

* Use new liveview used_input? function

* Alpine improvements

* Add select input to storybook

* Bring back `render_form` for CRM

* Configure eslint so it can see deps

* Remove LiveViewTest patch

* Fix test for phoenix liveview 1.0

* Build assets in prod

* Fix tests

* Attempt to fix lint error

* Add explicit text color to input

* mix format

* Format after merge master

* Add moduledocs

* Only run storybook in production

* Update storybook dependency

* Mix format
2025-01-13 12:31:18 +00:00
ruslandoga 15a38cc580
revert to sentry logger backend (#4919) 2024-12-18 18:57:49 +00:00
ruslandoga 24888cff3c
Drop Logger backends (#4862)
* drop console logger backend

* ah, switch Sentry too
2024-11-28 12:18:35 +00:00
Karl-Aksel Puulmann 2112feee88
CLICKHOUSE_DEFAULT_STORAGE_POLICY env (#4851)
In production we use `storage_policy = 'tiered'` by default but this is
not reflected in any migrations.

This change fixes that by introducing a new environment variable and
plumbing to be used in new (and old) migrations

Tested via setting env, doing `mix ecto.drop; mix ecto.create; mix ecto.migrate` and
checking resulting table schemas.
2024-11-26 12:54:17 +02:00
ruslandoga bef8b33087
update ecto_ch to fewer parens version (#4775) 2024-11-05 07:36:45 +00:00
Karl-Aksel Puulmann 62fb285b71
Comparisons: Fix exports issue (#4756)
* Increase max_parser_depth setting

This is currently causing issues in production with exports w/ comparisons, but
would blow up in other cases with sufficient data.

The reason this is now blowing up is that we create a large AND/OR condition
for breakdown comparisons. ecto-ch nests each conditional in its separate parentheses
causing us to reach the default 1000 max_parser_depth limit

We can improve this in a couple of ways:
- Not add unneccessary parentheses in ecto-ch
- Adding a new operator to make the AND/OR nesting require less conditionals, using
arrays and tuples instead

* Don't query comparisons for exports

csv exports don't actually output any comparison data, so querying for
it just causes issues.
2024-10-30 13:40:26 +00:00
ruslandoga ff3ee1ca3b
Improve boolean environment variables reading (#4632)
* improve bool env var reading

* add yes/no and tests
2024-10-16 07:11:53 +00:00
ruslandoga c8291639e5
default to invite_only in ce (#4616) 2024-10-04 04:29:13 +00:00