-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cuda heat example w quaditer #913
base: master
Are you sure you want to change the base?
Cuda heat example w quaditer #913
Conversation
end | ||
|
||
Kgpu = CUDA.zeros(dh.ndofs.x,dh.ndofs.x) | ||
gpu_dh = GPUDofHandler(dh) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe that's the plan anyway but wouldn't it be nicer to write Adapt
rules for DofHandler
and Grid
which return adapt_structure
for the GPU structs?
function Adapt.adapt_structure(to,dh:DofHandler)
return adapt_structure(GPUDofHandler(cu(Int32.(dh.cell_dofs)),GPUGrid(dh.grid)))
end
and then just use the "normal" structs from a user perspective that are automatically converted when needed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am still a bit split about this, because it contains a performance pitfalls. If we have repeated assembly, then the dof handler will be converted and copied to GPU for each assembly instead of only once.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Am I missing something or wouldn't the adapt call not also happen for each assembly kernel launch for the GPUDofHandler
in that case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should not happen, because we do not need to adapt the GPUDofHandler (it is already a GPU datastructure).
end | ||
|
||
|
||
gm = static_cellvalues.gm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe this is still a work in progress but the mix of functions and global variables make it kind of confusing to read the code.
What I did for now, and it's still work in progress:
This still work in progress and as my discussion with @termi-official last week I still need to work on the assembler, coloring algorthim. Some problems I have encountered that might be so straightforward to tackle:
|
Great to see some quick progress here!
I think that is straight forward to solve. We never really need the Dicts directly during assembly. We should be able to get away by just convert the Vectors (once) to GPUVectors and run the assembly with these. This might require 2 structs. One holding the full information (e.g. GPUGrid) and one which we use in the kernels (e.g. GPUGridView). Maybe the latter could be something like struct GPUGridView{TEA, TNA, TSA <: Union{Nothing, <:AbstractVector{Int}, <: AbstractVector{FaceIndex}, ..., TCA} <: AbstractGrid (?)
cells::TEA
nodes::TNA
subdomain::TSA
color::TCA
end where subdomain just holds the data which we want to iterate over (or nothing for all cells) and color is a vector for elements with one color of the current subdomain. |
A longer-term thing just to throw out the idea, but perhaps a more slim Grid could be nice? struct Grid{dim, C, T, CV, NV, S}
cells::CV
nodes::NV
gridsets::S
function Grid(cells::AbstractVector{C}, nodes::AbstractVector{Node{dim, T}}, gridsets) where {C, dim, T}
return new{dim, C, T, typeof(cells), typeof(nodes), typeof(sets)}(cells, nodes, gridsets)
end
end
struct GridSets
facetsets::Dict{String, OrderedSet{FacetIndex}}
cellsets::Dict{String, OrderedSets{Int}}
....
end allowing also |
I thought of this quite a bit already, whether we should have our grid in the form struct Grid{dim, C, T, CV, NV, S, TT}
cells::CV
nodes::NV
subdomain_info::S
function Grid(cells::AbstractVector{C}, nodes::AbstractVector{Node{dim, T}}, subdomain_info) where {C, dim, T}
return new{dim, C, T, typeof(cells), typeof(nodes), typeof(sets)}(cells, nodes, subdomain_info)
end
end where subdomain info contains any kind of subdomain information. This could also include potentially some optional topology information which we need for some problems. In the simplest case it would be just facesets and cellsets. However, we should do this in a separate PR. What do you think @fredrikekre ? |
Heat Example Prototype using CUDA.jl and StaticCellValues