* fix weekly/monthly email reports + stats report tests
* fix title of stats email reports
* add missing test
* do not render review installation link for consolidated view traffic drop emails + test
* refactor email_match_query to predicate fn
* make site_id_query_filter a public fn
* make traffic spike notifications work + test
* improve spike/drop notification email copies
* fix ScheduleEmailReports introduction comment
* add tests for stats report email scheduling
* remove space before .
* apply credo suggestion
* fix CE tests, use module attrs for text assertions
* skip frequency comment
* single query
* rename to site_member
* single query for real
* Upgrade phoenix and phoenix_live_view
* Use new phoenix controller options
* Run mix format
* Upgrade lazy_html and bcrypt
* Always put root layout
* Remove deprecated use Plug.Test
* Escpaing updates for LazyHTML
* Fix double layouts in site settings
* Fix more escaping in tests
* More LazyHTML updates
* Fix remaining LazyHTML issues
* More Floki -> LazyHTML conversions
* More Floki -> LazyHTML conversions
* Fix another issue with Floki -> LazyHTML
* Fix another lazyHTML issue
* Remove parsing from tree
* Fix HTML escaping in teams_test
* More test fixes
* More test fixes
* Add layout to browser_sso_notice pipeline
* Extract elem_count helper
* exclude consolidated views when counting/listing sites
* Use Teams.owned_sites/1 now that it excludes consolidated views
* Until we figure out better - make sites.consolidated virtual for CE
* Safeguard ingestion, update tests for gate keeper/site cache
* Ensure consolidated sites are excluded from helpscout counts
* Test that consolidated views are excluded from site listings
* Update tests for owned sites to ensure consolidated views are excluded
* Update CRM tests to exclude consolidated sites
* exclude consolidated sites from quota usage
* filter out conslolidated sites from deleting events background job
* exclude consolidated (change domain background job)
* retrieve non-consolidated sites only by default
* Ensure Site struct defines a default for `consolidated`
* Make `sites.consolidated` column available in CE
* Add missing test to reminder e-mails, consolidation is irrelevant here
* Do not send setup help e-mail for consolidated sites
* Ensure no setup success email is ever sent for consolidated sites
* Skip consolidated sites in legacy time on page cut off
* Add minor test to Sites.get_for_user/2
* Extend get_for_user/get_for_user! options signature
* Move test dir up
* Write tests for Sites.get_by_domain
* Minor DRY
* fix typos
* Provide Site.regular() and ConsolidatedView.sites() for querying
* Use Site.regular() in Repo.get_by() calls
* Fix typespecs
* Update missing Site.regular() filters
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* Replace usages of `Timex.to_unix` with native API
* Wrap call to `Timex.is_valid_timezone?`
* Wrap calls to `Timex.today(tz)`
* Replace `Timex.today()` with `Date.utc_today()`
* Replace `Timex.now()` with `DateTime.utc_now()`
* Replace `Timex.compare` with `Date.compare`
* Wrap `Timex.diff` calls
* Replace `Timex.Timezone.convert` with `DateTime.shift_zone!`
* Wrap `Timex.parse!`
* Replace `Timex.to_date` with native API calls
* Replace `Timex.beginning|end_of...` with native API calls for Date
* Wrap `Timex.beginning|end_of...` for DateTimes and Dates for years
* Replace `Timex.format(!)` with native API calls
* Replace `Timex.to_naive_datetime` with native API calls
* Wrap time humanizing routines using Timex
* Remove unnecessary `use Timex` instances
* Replace `Timex.shift` with native API calls
* Make `QueryParser.parse_date` handle gaps and ambiguities gracefully
* Replace `Timex.now(tz)` with `DateTime.now!(tz)`
* Use a more suitable Date function for comparison (h/t @aerosol)
* ✨
* wip
* wip
* Moduledoc false
* wip
* Update extra/lib/plausible/auth/sso/saml_config.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Include only data keys present in changes
* Improve audit logging for SSO domain verification
Make it more compact and hopefully more readable to CS
* Harden existing tests
* Use consistent naming
* Update audit entries migration: use UUIDs for primary keys
* Fix up tests
* Format
* Only test audit for EE
* Remove temporary String.Chars implementation
* Always log keys as per `derive` directive; include changes for inserts
* Write `actor_type` to audit entries
* Extract Audit.Repo functions
* Moduledocs
* Include change in audited deletions
* Make audit available only in EE build
A bit clunky? cc @zoldar
* Put test behind ee compilation flag
* Pin user e-mail in test
* Ensure encoder opts are passed for nested calls
* Carry `__allow_not_loaded__` even if no extractor defined
* Turn `actor_type` into an ecto enum type
* Remove unused function
* s/sso_forced/sso_force_mode_changed
* Unwrap single item list for protocol implementation
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Migration: audit entries (#5581)
* Migration: audit entries
* Put migration behind EE conditional
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* wip
* Fix up typespcs
* Extract domain status enum to module macros
* Cancel verification jobs when integration is removed
* Revisit verification interface
* Supply local DNS server for domain ownership testing
* Rename a test
* 👾
* Use identifier when submitting domain verification
* Disallow re-verification of already verified domains
* Revert "CRM: team (un)lock regardless of grace period (#5440)"
This reverts commit bb63c0d0e4.
* Implement grace period (un)lock just like in kaffy
* Test grace period handling
* Add `locked_by_admin` to teams schema
* Implement team locking
* Look up `teams.locked_by_admin` in relevant background services
* Add background servive tests
* Type faster than think!
* Purge bunny cdn cache when tracker script config is updated
* Only allow one purge task scheduled per site
Tested by adding logging and toggling a checkbox a few times. Only one job got executed.
* Typo
* add a new (feature flagged) upgrade page offering v5 plans
* include starter tier plans in available_plans_for + use dev prices in test
* upgrade page remodelling with starter tier
* mobile optimizations
* optimize for darkmode
* add embedded dashboards as a growth benefit
* do not hide header on LegacyChoosePlan
* consistent v5 plan feature order
* slight grandfathering notice adjustment
* display monthly price too on yearly plans
* default to v5 plans unlesss legacy? is true
* refactor: suggest volume not plan for emails
* align back link with page title
* render grandfathering notice for growth v4 too
* Append `__team` parameter to URLs in notification e-mails
* Improve copy in guest to team member promotion email
* Make "return_to" set in redirect in `RequireAccountPlug` actually work
* Improve accepted site transfer e-mail phrasing and link
* site.legacy_time_on_page_cutoff migration
* Update migration logic
* Cronjob for setting legacy_time_on_page_cutoff
* Test site creation default legacy_time_on_page_cutoff
* Use site.legacy_time_on_page_cutoff
Leaving the query param logic for now for testing purposes
* tweak cron
* Work around defaults in tests
* Fixes
* Make cutoff date in job configurable - useful for backfilling
* Update role enums with billing
* [wip] Ensure no team management can be done for roles other than admin/editor
* Track `current_role` at plug/mount level
* Some team role policy enforcement
* Guard remaining routes based on current team role
* Notify billing members via check usage worker
* Include billing members in trial/locked notifications
* Permissive team access policy when no current team assigned
Effectively the plug should no-op, so that existing user
notices/alerts can be displayed.
* Send cancellation notifications to billing role members
* Include billing members in renewal notifications
* Update moduledoc
* Exclude :guest role from team access auth plug
* Add tests for `Teams.get_or_create/1` and `Teams.get_by_owner/1`
* Start populating `current_team` in assigns fetching value from session
* Clean up team passing in invitation services
* Make site transfer service handle multi-team scenario
* Handle multi-team and permission transfer errors on controller level
* Handle multi-teams in site creation on service and controller level
* Drop validation limiting full membership to a single team
* Make user deletion account for public team ownership
* Adjust feature availability checks for Stats API key
* Use current_team when determining limits on site transfer invitation
* Adjust trial upgrade email submission to account for multiple owners
* Remove unnecessary `Teams.load_for_site/1`
* Spike renaming `owner` and `ownership` relationships to plural versions
* Make HelpScout integration handle owner of multiple teams gracefully
* Add FIXME note
* Resolve paddle callback issue by always provisioning a new team when none passed
* Set `current_team` as `my_team` only when user is an owner
* Implement basics of Teams CRM
* Extend Teams CRM
* Further adjust User and Site CRM and refine Team CRM
* Convert Enterprise Plan CRM to refer to team directly and not via user
* Remove unused virtual fields from User schema
* Add note to HelpScout integration
* Allow listing multiple owners under Site Settings / People
* Remove unused User schema relations
* Fix current team fetch in auth plug and context
* Implement basic team switcher
* Ensure (site) editor role is properly handled in site actions auth
* Don't set `site_limit_exceeded` error marker on `permission_denied` error
* Link from HS integration to Team CRM instead of User CRM when available
* Ensure consistent ordering of preloaded owners
* Add `with_subscription` preload for optimisitation
* Add ability to search sites by team identifier
* Add ability to pick team when transferring ownership directly
* Fix failing HelpScout tests
* Scope by team when listing sites in dashboard and via API (optional)
* Add ability to search by team identifier in plans CRM lookup widget
* Add subscription plan, status and grace period to team status info
* Expose teams list in user CRM edit form and fix team details CRM view
* Fix Team Switcher styling
* Reorganise header nav menu
* Avoid additional queries when authenticating user
* Hide the pay/site transfer message on lock screen when teams FF is on
---------
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* Move `GracePeriod` under `Teams` and clean it up a bit
* Switch to relying on new team membership constraint when creating my team
* Remove unused unique_constraint from Teams.Membership schema changeset
* Comment out legacy fields and relationships
* WIP
* WIP 2
* WIP 3
* wip
* Remove teams backfill and consistency check scripts
* WIP 3
* Fix CheckUsage tests
* Update billing/subscription tests
* WIP 4
* Make site transfer fail if some invitation already exists
* Fixup: do symmetric invitation/site transfer check
* Update UI bugs: make listing sites/inviting admins work like before
* Fix Sites test
* Fix external sites controller test
* Fix live sites tests
* Fix props availability lookup
* Fix site controller tests
* Fix billing controller tests
* WIP - accept invitation tests
* Another round of test fixes + invitations logic bugs
* users_test -> teams_test
* Update registration via invitation
Here, we still rely on "polymorphic" invitation structures,
hence the "unified by id" helper.
For now, it'll remain local unless we discover it's
needed in the broader `Teams.Invitations` context.
cc @zoldar
* Yet another round of test and bugfixes along the way
* Include team in site setup success e-mail
* Fix send_site_setup_emails worker
* Fixed almost all tests except CRM ones
* Update enterprise plan admin test
* Fix CRM + remaining tests
* Address credo warnings (modulo one FIXME)
* Remove last FIXME and rephrase the invitation test case description
* Set Team fields via User CRM transparently
* Map user reference in Enterprise Plan CRM via team owner
* Fix resource actions in user CRM
* Get rid of warning when opening create form in API keys CRM
* Stop emitting warnings when editing Enterprise Plans via CRM
* Tests: Bump await_clickhouse_count interval
* Remove XXX marker
* Fix register from invitation link in email sent for ownership transfer
* Simplify fetching all pending site ownership site IDs
* Remove commented out schema fields
* Remove unused functions
* Address flakiness in ingest counter tests
* Remove unused `Teams.Sites.create`
* Don't restart trial on team with subscription when creating site
* Account for cases of legacy teams with empty trial expiry date
* Revert "Address flakiness in ingest counter tests"
This reverts commit 60dc1e4115.
* Fix flaky ingest counters tests under load
* Attempt 2
* Pre-emptively hardcode site ids in sampling cache test
to avoid supplying the same IDs alongside with counters test,
that inserts through another repo (async).
what we're observing is, clickhouse not summing mergetree columns fast
enough, even though we wait quite a bit.
* Fix ingest counter tests by accounting for delayed summation
---------
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* Pre-emptively introduce `site.team_owner` relation
* Drop null constraint on user_id from subscriptions and enterprise_plans
* Temporarily remove populating old schemas in Teams.Test
* Point to site.owner via new schema
* Switch more reads to teams schema WIP
* Fix AuhtorizeSiteAccess test
There's no need to translate `admin`<->`editor` here,
the redundancy is inlined wherever the plug is initialized.
* Fix regions test
* Fix main graph test
* Fix authorization test
* Try to rely on team for subscription/plans where applicable
* Test fixes
* Fix plans test
* Prep for CheckUsage changes
* Skip remaining CheckUsage tests for now
* Fix user deletion to account for team relations
* Fix HelpScout tests
* 💀 Modify ingestion to read team schemas
* Made all tests green except skipped ones
* Mute warnings about transferring site with no order
By making artificial site membership struct,
when reading data off team membership schema.
* Fix site removal test case
* Re-enable locked site tests, that don't have to rely on `SiteLocker`
* Format
* Revert "Mute warnings about transferring site with no order"
This reverts commit 0e45f8c9d9.
* Re-enable old models and fix remaining tests
* Use new factories in a long running minio test
* FIXME->TODO
* Fix remaining tests in legacy mode (no FF raised)
* oof
cc @zoldar
* Add missing definitions of editor role in FE code
* Remove no longer relevant comment about roles
* Fix JS formatting
* Always prioritize site transfers over memberships in sites list
* Fix misaligned "Reject" invitation button
* Fix site pinning when user is guest in multiple sites in team
* Fix subscription settings controller tests
---------
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* Move `bulk_transfer_ownership_direct` under `AcceptInvitation`
* [WIP] Switch ownership transfer operations to read from team schemas behind FF
* Fix usage test regression
* Semantics - current user; ownership is not necessarily involved
* Perform remaining read via adapter; remove obsolete test
* Properly list site with pending site transfer while being guest on a team
* Account for pending site transfers in Settings > People list
---------
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* Switch on teams schema in choose plan view behind FF
* Proxy via Read adapter where applicable for billing context
* Proxy remaining plan-related functions
* Switch enterprise_configured?/1 tests to use the adapter
* Format
* Update SiteLocker tests
* Actually use `has_active_subscription?/1` billing adapter
---------
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* Clean site transfers after 48 hours
* Sync accepting site transfers and invitations within transaction
* Add dry run mode to teams backfill and make it a default
* Extend invitation clean worker tests
* Ensure team is present before use in sync logic
* Ensure teams backfill works against partially assigned sites
* Associate site with team on creation
* Associate site with team on sync
* Reuse alias
* Add tests for invitation creation sync
* Move team assertions to a helper module
* Format
* Test team creation on site creation via Sites context module
* Add tests for teams sync on subscription changes
* Tag tests
* Test grace period start syncing up with teams
* Test grace period manual lock sycning w/ teams
* Test grace period end sycing up w/ teams
* Test clearing grace period sync with teams
* Update moduledoc
* Fix missing preloads and wrong result pattern matching in sync logic
* Test sync on accepting invites and site transfers
* Test sync on membership role update and member removal
* transfer async fix WIP
* Stop privisioning team in site factory
* Remove unused relationship from Site schema
* Ensure consistent parsing of `passthrough` from Paddle webhook
* Update team passthrough notification tests & logic
---------
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* Extend schemas with new fields and relationships for teams
* Implement listing sites and sites with invitations with teams
* Implement creating invitations with teams
* Implement accepting invites with teams
* Add `Teams.SiteTransfer` schema
* Implement creating ownership transfers
* Implement accepting site transfer between teams
* Make results shapes from `Teams.Memberships` role functions more consistent
* Remove :team relation from ApiKey schema
* Pass and provision team on subscription creation
* Pass and provision team on enterprise plan creation
* Implement creating site for a team
* Keep team in sync during legacy ownership transfer and invitations
* Resolve conflict in `Teams.get_or_create` without transaction
* Abstract `GracePeriod` manipulation behind `Plausible.Users`
* Put `User.start_trial` behind `Plausible.Users` API
* Sync team fields on user update, if team exists
* Sync cleaning invitations, updating and removing members
* Transfer invitations too
* Implement backfill script
* Allow separate pg repo for backfill script
* Rollback purposefully at the end
* Update backfill script with parallel processing
* Use `IS DISTINCT FROM` when comparing nullable fields
* Handle no teams to backfill case gracefully when reporting
* Parallelize guest memberships backfill
* Remove transaction wrapping and query timeouts
* Make team sync check more granular and fix formatting
* Wrap single team backfill in a transatction for consistent restarts
* Make invitation and site transfer backfills preserve invitation ID
* Update migration repo config for easier dev access
* Backfill teams for users with subscriptions without sites
* Log timestamps
* Put teams sync behind a compile-time flag
* Keep timestamps in sync and fix subscriptions backfill
* Fix formatting
* Make credo happy
* Don't `use Plausible.Migration` to avoid dialyzer complaining
None of the tooling from there is used anywhere and `@repo` can
be defined directly in the migration script.
* Drop SSL workarounds in the backfill script
---------
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* Turn .eex templates into .heex
* Add new compile-time presets to `PlausibleWeb`
* Fix remaining templates
* Update static components
* Update live components
* Update live views
* Update rest of the owl
* Update mjml template
* Format
* Format
* Revert MJML stuff, it's coupled with EEx
* yawn at test
* Get rid of `FormHelpers` module
* Ensure YOU label shows up first on IP rules list
* Update lib/plausible_web/templates/email/welcome_email.html.heex
Co-authored-by: Artur Pata <artur.pata@gmail.com>
* Fix create site email link
* Fix server error markup (and turn thanks into heex)
* Format
---------
Co-authored-by: Artur Pata <artur.pata@gmail.com>
* Check usage for users with active/cancelled/past due subscriptions
So that over limit e-mails and grace period is handled
regardless - they're still subscribers.
* Ensure current subscription is fetched, for an ongoing plan
* Add extra test
* Revert "Ensure current subscription is fetched, for an ongoing plan"
This reverts commit 9ddc7bdccb.
* Fixup test
* Reapply "Ensure current subscription is fetched, for an ongoing plan"
This reverts commit 7ab537954b.
* Do a subscription preload and assert ID match
* Use inner lateral join to fetch subscriptions
* Turn `Plausible.Auth.UserSession` into full schema
* Implement token based sessions and use them as default
* Ignore expired user sessions during retrieval from DB
* Implement plug bumping user session last used and timeout timestamps
* Implement Oban worker removing expired user sessions with grace period
* Implement legacy session conversion on touch, when applicable
* Update `UserAuth` moduledoc
* Extend `UserAuth` tests to account for db-backed session tokens
* Update CHANGELOG
* Add tests for `UserSessionTouch` plug
* Add test for `CleanUserSessions` worker
* Add logging of legacy session retrievals
* Use single update permitting stale records when touching user session
* Don't fetch session and user for external API endpoints (/api/event too)
* Refactor `Users.with_subscription/1` and expose helper query
* Skip fetching session in legacy `SessionTimeoutPlug`
* Rely on user session assign from `AuthContext` in `SentryContext`
* Silence legacy session warnings in `UserSessionTouchTest`
* Rely on session assign from `AuthPlug` in `SuperAdminOnlyPlug`
* Change `UserAuth` to get session, user and last subscription in one go
* Avoid refetching user session in `AuthorizeSiteAccess` plug
* Fix code formatting
* Refactor `UserAuth.get_user_token/1` (h/t @aerosol)
* Remove bogus empty opts from `scope` declarations in router
* Only touch session once an hour and keep `user.last_seen` in sync
* Bring back logging of legacy token use
* Migration: add installation meta
* Update site schema with installation meta
* Remove VERIFICATION_ENABLED env var
* Add context API to create/remove special goals
* Add context api to update installation meta
* Remove verification enabled check
* Update new progress flow definitions
* Update generic components
* Remove internal /status API
* Implement installation live view
* Update traffic change notifier link
* Update verification, no more modal
* Update routes
* Remove focus.html - will unify everything under app layout
* Fix broken link
* Update templates with focus_box mostly
* Update controller tests
* Update controllers and stop using the focus layout
* copy changes
* Update verification.ex
* Remove dead template
* Update settings_general.html.heex
* Update copy in tests
* Update installation.ex
* Remove dangling dot
* Fix link
* Update installation.ex
* Update installation.ex
* Better tooltips?
* Simpler labels
* Revert "Simpler labels"
This reverts commit 797560ef82f2067458b03b884be5aecc8fdc72bc.
* Add copy to clipboard link and fix snippet's dark mode
* Offer installation detection skip only if ws connected
* Put COPY link at the bottom with background
* Make tooltips link to docs
* Fix cherry-pick gone wrong
* Hide tooltips on mobile screens
* WIP: 404 tracking wizard
* Revert "WIP: 404 tracking wizard"
This reverts commit a9c9c79bbd.
* Update lib/plausible_web/live/components/verification.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Update lib/plausible_web/live/installation.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Use current_user from socket.assigns
* Update lib/plausible_web/live/installation.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Use current_user from socket.assigns
* Use conn.private to steer verification tests
* Drop non-sticky tooltip in favour of component parametrization
Co-authored-by: Artur Pata <artur.pata@gmail.com>
* Reapply "WIP: 404 tracking wizard"
This reverts commit 3ba81671d7.
* Fix installation tests including 404 tracking
* Fixup the tooltip component
* Format
* Update installation.ex
* Put flash whenever installation option changes
* Use last known installation type on domain change
* Extract user flow definition to provide compile-time checks
* See if this helps running CE migrations successfully
* Use `styled_link` on registration/login views
* Don't crash when there's no conn.private carried over
* Format
* Push "Determining installation type" message a bit lower
* Use links and footer lists uniformly
This commit introduces a `<.focus_list/>` component
for rendering focus box footer links with colored
discs. It also equips generic link components
with the ability of sending non-GET requests
along with CSRF token, so we can apply uniform
styling and stop using legacy Phoenix link tags.
cc @zoldar @apata
* ws 👾
* Render more descriptive flashes on script config change
---------
Co-authored-by: Marko Saric <34340819+metmarkosaric@users.noreply.github.com>
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
Co-authored-by: Artur Pata <artur.pata@gmail.com>
* Expose current visitors 12h aggregate
* Remove unused site association
* Distinct drop/spike notification factories
* Rename modules accordingly + implement drop handling
* Rename periodic oban service
* Implement drop email
* Rest of the owl
* Update changelog
* Update moduledoc
* Update moduledoc
* Min threshold to 1
* Threshold 1
* Remove merge artifact
* Put panel behind a feature flag
* Format
* pull last_bill_date from paddle sandbox in mix task
* move cycle usage checks to Quota module
* move quota.ex to a subfolder
* split up Quota module
* set choose-plan pageview slider according to usage
* silence credo
Adam Rutkowski
RobertJoonas
Uku Taht
Adrian Gruntkowski
Marko Saric
Karl-Aksel Puulmann
ruslandoga