Add mm map

[jlaitine]

Summary

This PR adds a simple linked list to store mmap mappings, to support unmap.

The drivers which want to store mappings can use mm_map_add and mm_map_rm to register/unregister mappings to the task_group.

This PR also takes the mm_map into use in mm/shm driver and anonymous mappings. The sys-v shm interface still maintains it's own granule allocator in task_group

This PR also takes the mm_map into use in "rammap"

Impact

Adds interface to make it possible to do munmap
Adds munmap for anonymous mappings
Fixes mm/shm for process exit; previously it doesn't free the reserved physical pages if they are mapped at process exit

Testing

Tested in stm32 and mpfs riscv builds, together with inhouse posix shmfs driver

[jlaitine]

There were some issues left after re-base. I commented them, and will fix those tomorrow.

[xiaoxiang781216]

BTW, it's better to migrate the ram map and anon map to the new interface, which is clearer and also demo that new mmap interface is flexible enough to support the complex case.

[jlaitine]

BTW, it's better to migrate the ram map and anon map to the new interface, which is clearer and also demo that new mmap interface is flexible enough to support the complex case.

Agree. I can move ram mapping (MAP_PRIVATE, MAP_ANONYMOUS) to an own file at least. It should also have an own CONFIG flag (can make it default to y to keep current functionality). I find this interface quite useless as it is now, it doesn't even support mapping from page pool.

rammaps deserve an own PR, but I'll see if I can make use of the mm_map in it already as you suggested above. That sould be extended to also support MAP_SHARED. I find this interface quite useless as it is now, and this is why I don't like touching it too much in this PR.

I think that after we agree on the interface for storing the mappings, I will present the posix SHMFS; this is an interface that I know is interesting to many. For example we have implemented an own version of PX4 uORB publish subscribe interface with this, and it is really nice, since it works inter process both in linux and nuttx (in all flat, protected and kernel). There are also other pubsub interfaces which could utilize this.

[xiaoxiang781216]

BTW, it's better to migrate the ram map and anon map to the new interface, which is clearer and also demo that new mmap interface is flexible enough to support the complex case.

Agree. I can move ram mapping (MAP_PRIVATE, MAP_ANONYMOUS) to an own file at least. It should also have an own CONFIG flag (can make it default to y to keep current functionality). I find this interface quite useless as it is now, it doesn't even support mapping from page pool.

Yes, but it's useful for no MMU case. it's a first step to incorporate rammap into the new mmap interface. Once it's done, we can support page pool for MMU.

rammaps deserve an own PR, but I'll see if I can make use of the mm_map in it already as you suggested above. That sould be extended to also support MAP_SHARED. I find this interface quite useless as it is now, and this is why I don't like touching it too much in this PR.

supporting MAP_SHARED require we save some mapping info to inode, but I think it's very important feature to make rammap usefull.

I think that after we agree on the interface for storing the mappings,

Yes, I think so.

I will present the posix SHMFS; this is an interface that I know is interesting to many. For example we have implemented an own version of PX4 uORB publish subscribe interface with this, and it is really nice, since it works inter process both in linux and nuttx (in all flat, protected and kernel). There are also other pubsub interfaces which could utilize this.

Nice feature. Where can I find the document or source code? is it bas on PX4 uORB or this new one: #6653

[jlaitine]

I will present the posix SHMFS; this is an interface that I know is interesting to many. For example we have implemented an own version of PX4 uORB publish subscribe interface with this, and it is really nice, since it works inter process both in linux and nuttx (in all flat, protected and kernel). There are also other pubsub interfaces which could utilize this.

Nice feature. Where can I find the document or source code? is it bas on PX4 uORB or this new one: #6653

It is based on PX4 uORB; the project that I am working with atm. is drone project and the uORB needs to be a drop-in replacement for PX4

[jlaitine]

BTW, it's better to migrate the ram map and anon map to the new interface, which is clearer and also demo that new mmap interface is flexible enough to support the complex case.

Added patches for this

[jlaitine]

The rammaps went broken, I will need to fix it still. It was a bit too quick...

[jlaitine]

now done. Rammaps doesn't need the private mapping structure for anything, all info fits in map_entry_s. So removed that too.

[jlaitine]

re-based to latest master

[jlaitine]

One way to remove group argument from mm_map_remove is just to put a restriction that "driver mustn't call mm_map_remove in the munmap function if the group is unknown (NULL)". Then you would get rid of the iffy goto in mm_map_remove function as well. All entries can be deleted directly in mm_map_destroy() regardless of the current context.

[jlaitine]

@xiaoxiang781216 : Here is a commit which shows how the IF would look like if you added the mm_map_s as an argument to every function (the option 1. you mentioned avove):

tiiuae@1385bca

In my opinion it just complicates things for every driver using the interface, since they need to fetch & dig into the task_group structure, even if they don't need it for anything else.

But if you think that this would look better, I am fine with it.

Please let me know which way you insist this to be, I will then clean up the git history accordingly.

[xiaoxiang781216]

One way to remove group argument from mm_map_remove is just to put a restriction that "driver mustn't call mm_map_remove in the munmap function if the group is unknown (NULL)". Then you would get rid of the iffy goto in mm_map_remove function as well. All entries can be deleted directly in mm_map_destroy() regardless of the current context.

My comment isn't to remove group from mm_map_remove, but make the usage consistent.

@xiaoxiang781216 : Here is a commit which shows how the IF would look like if you added the mm_map_s as an argument to every function (the option 1. you mentioned avove):

tiiuae@1385bca

In my opinion it just complicates things for every driver using the interface, since they need to fetch & dig into the task_group structure, even if they don't need it for anything else.

But if you think that this would look better, I am fine with it.

Please let me know which way you insist this to be, I will then clean up the git history accordingly.

Yes, I think it isn't good to let's driver dig into task_group struct. Let's come back to my first suggestion but reject by you:
All mm_map_s related function use tasK_group_s as the first argument.

[jlaitine]

Yes, I think it isn't good to let's driver dig into task_group struct. Let's come back to my first suggestion but reject by you: All mm_map_s related function use tasK_group_s as the first argument.

Any other part of the task_group_s doesn't belong to mm_map code, so it is really better to just pass mm_map_s where it is needed. I decoupled this from task_struct internals by adding helper macros into sched.h

[jlaitine]

re-based to current master

[xiaoxiang781216]

LGTM.