[WHY]
Function "amdgpu_irq_update()" called by "amdgpu_device_ip_late_init()" is an atomic context.
We shouldn't access registers through KIQ since "msleep()" may be called in "amdgpu_kiq_rreg()".
[HOW]
Move function "amdgpu_virt_release_full_gpu()" after function "amdgpu_device_ip_late_init()",
to ensure that registers be accessed through RLCG instead of KIQ.
Call Trace:
<TASK>
show_stack+0x52/0x69
dump_stack_lvl+0x49/0x6d
dump_stack+0x10/0x18
__schedule_bug.cold+0x4f/0x6b
__schedule+0x473/0x5d0
? __wake_up_klogd.part.0+0x40/0x70
? vprintk_emit+0xbe/0x1f0
schedule+0x68/0x110
schedule_timeout+0x87/0x160
? timer_migration_handler+0xa0/0xa0
msleep+0x2d/0x50
amdgpu_kiq_rreg+0x18d/0x1f0 [amdgpu]
amdgpu_device_rreg.part.0+0x59/0xd0 [amdgpu]
amdgpu_device_rreg+0x3a/0x50 [amdgpu]
amdgpu_sriov_rreg+0x3c/0xb0 [amdgpu]
gfx_v10_0_set_gfx_eop_interrupt_state.constprop.0+0x16c/0x190 [amdgpu]
gfx_v10_0_set_eop_interrupt_state+0xa5/0xb0 [amdgpu]
amdgpu_irq_update+0x53/0x80 [amdgpu]
amdgpu_irq_get+0x7c/0xb0 [amdgpu]
amdgpu_fence_driver_hw_init+0x58/0x90 [amdgpu]
amdgpu_device_init.cold+0x16b7/0x2022 [amdgpu]
Signed-off-by: Chong Li <chongli2@amd.com>
Reviewed-by: JingWen.Chen2@amd.com
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
add jpeg table size to ctx table size rather than override it
Signed-off-by: Jane Jian <Jane.Jian@amd.com>
Reviewed-by: JingWen Chen <JingWen.Chen2@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add some basic definitions and structure member. Inscrease MAX_WB slots
to 1024 to support the increasing number of rings for multiple partitions.
v2: unify naming style
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
It looks better to place this field in ring
structure. Also drop the repeated ring funcs definitions
if there's no difference except for vmhub field.
v2: rename the field to vm_hub like others (Le)
v3: apply the changes to new ip blocks (Hawking)
v4: fix vcn sw ring (Alex)
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add common IP handling for GC 9.4.3
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <Le.Ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
init asic funcs and cp/pg flags for GC 9.4.3
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <Le.Ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
add gfx_funcs callbacks implemenation based on
gc_v9_4_3 ip headers
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Reserve the MOUDLE_FIRMWARE declaration of gc_11_0_*_mes.bin
to fix falling back to old mes bin on failure via autoload.
Fixes: 97998b893c ("drm/amd/amdgpu: introduce gc_*_mes_2.bin v2")
Signed-off-by: Li Ma <li.ma@amd.com>
Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Due to switch to golden tsc register to get clock counter for raven/ raven2.
Chang the reference clock from 25MHZ to 100MHZ.
Suggested-by: shanshengwang <shansheng.wang@amd.com>
Reviewed-by: Evan Quan <evan.quan@amd.com>
Signed-off-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
GFX is in gfxoff mode during s0ix so we shouldn't need to
actually execute kfd_iommu_suspend/kfd_iommu_resume operation.
Signed-off-by: Aaron Liu <aaron.liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
all the gc v9_4_3 registers fall in gc_rlcpdec address range
have different relative offsets and base_idx from the ones
defined in gc v9_0 ip headers. gc_v9_0_rlc_funcs can not be
reused anymore for gc v9_4_3
v2: drop unused handshake function (Alex)
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <Le.Ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Was introduced as workaround. not needed anymore
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Jack Gui <Jack.Gui@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This patch adds double include protection for doorbell.h
Cc: Christian Koenig <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian Koenig <christian.koenig@amd.com>
Signed-off-by: Shashank Sharma <shashank.sharma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Rename doorbell.num_doorbells to doorbell.num_kernel_doorbells to
make it more readable.
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian Koenig <christian.koenig@amd.com>
Acked-by: Christian Koenig <christian.koenig@amd.com>
Signed-off-by: Shashank Sharma <shashank.sharma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Due to raven/raven2 maybe enable sclk slow down,
they cannot get clock count by the RLC at the auto level of dpm performance.
So switch to golden tsc register.
Suggested-by: shanshengwang <shansheng.wang@amd.com>
Reviewed-by: Evan Quan <evan.quan@amd.com>
Signed-off-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Rework retry fault removal from the software filter by
storing an expired timestamp for a fault that is being removed.
When a new fault comes, and it matches an entry in the sw filter,
it will be added as a new fault only when its timestamp is greater
than the timestamp expiry of the fault in the sw filter.
This helps in avoiding stale faults being added back into the
filter and preventing legitimate faults from being handled.
Suggested-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Philip Yang <Philip.Yang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This patch enables the IH retry CAM on GFX9 series cards. This
retry filter is used to prevent sending lots of retry interrupts
in a short span of time and overflowing the IH ring buffer. This
will also help reduce CPU interrupt workload.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
All chips that support RAS also support IP discovery, so
use the IP versions rather than a mix of IP versions and
asic types. Checking the validity of the atom_ctx pointer
is not required as the vbios is already fetched at this
point.
v2: add comments to id asic types based on feedback from Luben
Reviewed-by: Luben Tuikov <luben.tuikov@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: Luben Tuikov <luben.tuikov@amd.com>
Enable SDMA queue empty context switching. SDMA context switch due to
quantum programming no longer done here (as of sdma v6), so re-name
sdma_v6_0_ctx_switch_enable to sdma_v6_0_ctxempty_int_enable to reflect
this.
Also program SDMAx_QUEUEx_SCHEDULE_CNTL for context switch due to
quantum in KFD. Set to amdgpu_sdma_phase_quantum (defaults to 32 i.e.
3200us).
Signed-off-by: Graham Sider <Graham.Sider@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Reviewed-by: Stanley Yang <Stanley.Yang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
CP_HQD_HQ_STATUS0[29] bit will be used by CPFW to acknowledge whether
PCIe atomics are supported. The default value of this bit is set
to 0. Driver will check whether PCIe atomics are supported and set the
bit to 1 if supported. This will force CPFW to use real atomic ops.
If the bit is not set, CPFW will default to read/modify/write using the
firmware itself.
This is applicable only to GFX11 RS64 CP with MEC FW >= 509. If MEC
FW < 509 and for all GFX11 F32 CP, PCIe atomics needs to be supported
else it will skip the device.
This commit also involves moving amdgpu_amdkfd_device_probe() function
call after per-IP early_init loop in amdgpu_device_ip_early_init()
function so as to check for RS64 enabled device.
Signed-off-by: Sreekant Somasekharan <sreekant.somasekharan@amd.com>
Reviewed-by: Graham Sider <Graham.Sider@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
XGMI RAS should be according to the gmc xgmi physical nodes number,
XGMI RAS should not be enabled if xgmi num_physical_nodes is zero.
Signed-off-by: Stanley.Yang <Stanley.Yang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Aldebaran supports VCN and JPEG RAS, it reports unexpected
block id message during VCN and JPEG RAS initialization if VCN
and JPEG block id not defined.
Signed-off-by: Stanley.Yang <Stanley.Yang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This achieves the same result as the sequence used in emit_flush_gpu_tlb
but the invalidation is now a single packet instead of the 3 packets
required to implement reg_write_reg_wait.
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[why]
regGOLDEN_TSC_COUNT_LOWER/regGOLDEN_TSC_COUNT_UPPER are protected and
unaccessible under sriov.
The clock counter high bit may update during reading process.
[How]
Replace regGOLDEN_TSC_COUNT_LOWER/regGOLDEN_TSC_COUNT_UPPER with
regCP_MES_MTIME_LO/regCP_MES_MTIME_HI to get gpu clock under sriov.
Refine get gpu clock counter method to make the result more precise.
Signed-off-by: Tong Liu01 <Tong.Liu01@amd.com>
Acked-by: Luben Tuikov <luben.tuikov@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For DMA-MAP userptr on other GPUs, the dma address array
will be populated in amdgpu_ttm_backend_bind.
Remove the redundant call from the driver.
v2:
update the comment
Signed-off-by: Shane Xiao <shane.xiao@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For SG BO to DMA-map userptrs on other GPUs, the SG BO
need inherit MTYPEs in PTEs from original BO.
If we set the flags, the device can be coherent with
the CPUs and other GPUs.
v2:
1. Drop unnecessary flags check
2. Remove local variable align
Signed-off-by: Shane Xiao <shane.xiao@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For userptr bo with iommu on, multiple GPUs use same system
memory dma mapping address when both adev and bo_adev are in
identity mode or in the same iommu group.
If RAM direct map to one GPU, other GPUs can share the original
BO in order to reduce dma address array usage when RAM can
direct map to these GPUs. However, we should explicit check
whether RAM can direct map to all these GPUs.
This patch fixes a potential issue that where RAM is
direct mapped on some but not all GPUs.
v2:
1. Update comment
2. Add helper function reuse_dmamap
Signed-off-by: Shane Xiao <shane.xiao@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The driver doesn't resubmit jobs on hangs any more, hence drop
the hang limit parameter - amdgpu_job_hang_limit, wherever it is used.
Suggested-by: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Mario Limonciello <mario.limonciello@amd.com>
Cc: Kent Russell <kent.russell@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[Why]
As MES KIQ is dequeued, tell RLC that KIQ is inactive
[How]
Clear the RLC_CP_SCHEDULERS Active bit which RLC checks KIQ status
In addition, driver can halt MES under SRIOV when unloading driver
v2:
Use scheduler0 mask to clear KIQ portion of RLC_CP_SCHEDULERS
Signed-off-by: Yifan Zha <Yifan.Zha@amd.com>
Reviewed-by: Horace Chen <horace.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[Why]
Need dequeue MES KIQ under SRIOV when unloading driver
[How]
Modify mes_v11_0_kiq_dequeue_sched which was used to dequeue MES SCHED
to support veriable pipe.
Add MES KIQ dequeue in hw fini
Signed-off-by: Yifan Zha <Yifan.Zha@amd.com>
Reviewed-by: Horace Chen <horace.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
To avoid new mes fw running with old driver, rename
mes schq fw to gc_*_mes_2.bin.
v2: add MODULE_FIRMWARE declaration
v3: squash in fixup patch
Signed-off-by: Jack Xiao <Jack.Xiao@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Skip mode2 reset only for IMU enabled APUs when do S4.
This patch is to fix the regression issue
https://gitlab.freedesktop.org/drm/amd/-/issues/2483
It is generated by commit b589626674 ("drm/amdgpu: skip ASIC reset
for APUs when go to S4").
Fixes: b589626674 ("drm/amdgpu: skip ASIC reset for APUs when go to S4")
Link: https://gitlab.freedesktop.org/drm/amd/-/issues/2483
Tested-by: Yuan Perry <Perry.Yuan@amd.com>
Signed-off-by: Tim Huang <tim.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For GC 9.4.3, disable retry as default and XNACK can be different
modes per process.
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Fix backward compatibility issue to stay with
the old name of xgmi_wafl node.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Tao Zhou <tao.zhou1@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[Why]
Reset(mode1) failed as JPRG IP did not reinit under sriov.
[How]
Add JPEG IP block to sriov reinit function.
Signed-off-by: Yifan Zha <Yifan.Zha@amd.com>
Reviewed-by: Horace Chen <Horace.Chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Global variables do not need to be initialized to 0 or NULL and checkpatch
flags this error in drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c:
ERROR: do not initialise globals to NULL
+char *amdgpu_disable_cu = NULL;
+char *amdgpu_virtual_display = NULL;
Fix this checkpatch error.
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Mario Limonciello <mario.limonciello@amd.com>
Cc: Hamza Mahfooz <Hamza.Mahfooz@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add GMC IP handling for GC 9.4.3
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <Le.Ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Initialize gfxhub1.2 and mmhub1.8 function calls
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Hack the mmhub 1.7 reg offset for initial support
v2: squash in header switch, CG funcs (Alex)
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Hack the gc 9.0 reg offset for initial support
v2: squash in header switch (Alex)
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Initialize various gmc sw/hw settings/configurations
for GC 9.4.3.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <Le.Ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Set family for GC 9.4.3
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <Le.Ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
switch to the new nbio generation for NBIO 7.9.0.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <Le.Ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
v7_9 is a new nbio generation ip.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <Le.Ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Chong Li
Jane Jian
Le Ma
Hawking Zhang
Li Ma
Jesse Zhang
Aaron Liu
Shashank Sharma
YiPeng Chai
Mukul Joshi
Alex Deucher
Graham Sider
Sreekant Somasekharan
Stanley.Yang
Pierre-Eric Pelloux-Prayer
Lijo Lazar
Tong Liu01
lyndonli
Shane Xiao
Srinivasan Shanmugam
Yifan Zha
Jack Xiao
Tim Huang
Amber Lin