RFC - New src folder structure "module oriented"

[Vadorequest]

There had been a few feedbacks regarding the folder structure of the src folder.

The goal is to regroup code that is related to each other into modules, represented as an unified folder. Currently, the code is cluttered between "components", "utils" and "types" folders

Current structure:

New structure - Proposal 1:

This structure is theoretical (WIP)

Store everything that is common across pages or too wide to belong into a dedicated module into a common folder. Config is also stored into a dedicated folder (instead of src/). Then, each module is a folder under src and contains all files related to the module. It can, optionally, contain its own utils/types subfolders if it's too big.

	src
		common
			components
			hocs
			hooks
			stores
			types
			utils
			server
				middlewares
		config
			airtableSchema.ts
			constants.ts
			i18nConfig.js
		pages
			(same as before)
		// modules
		fetchAirtableDataset
			types
			utils
		amplitude
			types
			utils
		api
			types
			utils
		airtableApi
			types
			utils
		vercelCache
			types
			utils
		cookieManager
			types
			utils
		tests
			env.test.ts
			extend-jest
			tests-mocks.ts
			mocks
		sentry
		nextjs
		...

New structure - Proposal 2:

This structure is theoretical (WIP)

Similar to "Proposal 1", but this time everything is under src/modules (except for pages), which makes it slightly obvious that the approach is module-based.

Similarly, we could have "modules", "common" and "pages" under src. It'd be okay too, and could make more sense (let's call it "Proposal 3").

	src
		modules
			common
				components
				hocs
				hooks
				stores
				types
				utils
				server
					middlewares
			config
				airtableSchema.ts
				constants.ts
				i18nConfig.js
			fetchAirtableDataset
				types
				utils
			amplitude
				types
				utils
			api
				types
				utils
			airtableApi
				types
				utils
			vercelCache
				types
				utils
			cookieManager
				types
				utils
			tests
				env.test.ts
				extend-jest
				tests-mocks.ts
				mocks
			sentry
			nextjs
			...
		pages
			(same as before)

New structure - Proposal 3:

This structure is theoretical (WIP)

Similar to "Proposal 2", but this time "modules", "common" and "pages" under src. It'd be okay too, and could make more sense because "common" isn't really much of a module.

	src
		common
			components
			hocs
			hooks
			stores
			types
			utils
			server
				middlewares	
		modules
			config
				airtableSchema.ts
				constants.ts
				i18nConfig.js
			fetchAirtableDataset
				types
				utils
			amplitude
				types
				utils
			api
				types
				utils
			airtableApi
				types
				utils
			vercelCache
				types
				utils
			cookieManager
				types
				utils
			tests
				env.test.ts
				extend-jest
				tests-mocks.ts
				mocks
			sentry
			nextjs
			...
		pages
			(same as before)

---

What are your thoughts on this? My personal preference goes for "Proposal 3".

[Vadorequest]

@jajourda @sovattha @samuelcastro @Demmonius Do you have any preference about this?

[sovattha]

Definitely proposal 3 for me. Thanks for asking!

I did not use "modules" much so far, so hiding this complexity away is good for a beginner.

I like the fact that the most important folders are right under src. For me: common, pages, components.

[Vadorequest]

I did not use "modules" much so far, so hiding this complexity away is good for a beginner.

There is no "modules" folder yet, it's a new thing that basically regroup what's currently split between "components", "utils" and "types" (and maybe more).

The thinking it that as soon as you know some code is related and can be grouped together, then you can do so within "modules". Until then, you can do so in "common".

This way, you don't have to think hard when starting something new and just go with "common", and then if at some point it feels right to group together some files then you can create a dedicated module for it.

I believe it'll be easier to maintain this way, because code that relates together will be in a single folder and easier to locate.

[samuelcastro]

Definitely Proposal 3, this is what I'm using, although I put everything under common but I like the modules folder.

I proposed something similar before: #44

[Vadorequest]

Yes, I want to do it. I just lack time and don't know when it'll happen.

[Vadorequest]

I'm planning to work on this, this week.

[samuelcastro]

Hey @Vadorequest I'm wondering if this is something you're already working? Because since there are a lot of new changes recently, I don't want to merge them and then need to merge a big change of the folder structure again.

If you think this might not get implemented soon, I'm just going to merge the new changes in anyway.

[Vadorequest]

On the contrary, I'd rather advise not to do it all at once, it might be too complicated.

You'll probably have a better time merging the current changes now and this later.

And no, I'm not yet working on it. But I'll definitely be doing it this month (🤞 )

[samuelcastro]

Ok, thanks for the update.

[Vadorequest]

Work has started on #264

@sovattha @samuelcastro Could you review and give your feedback? Does it look like what you expected?

I'm also considering using @ when loading paths. E.g: import Button from '@/components/button'

https://nextjs.org/docs/advanced-features/module-path-aliases

It would be more resilient to moving files around, by not creating conflicts within the file itself. And it'd avoid breaking paths when moving files around (as it happened to me a lot today, had to fix those manually afterwards)

I've never used module path aliasing, do you think it's a good idea to integrate it now? Would it make the upgrade smoother for you?

[Vadorequest]

Updated #265 with support of module aliases for .js files too, and used them in Cypress and Storybook as well.

[Vadorequest]

@sovattha @samuelcastro Everything looks good to me. I'm awaiting PR review for both #265 and #264. I could merge them right away (they're both stable) but considering how big those changes are I'd very much prefer getting feedback, especially from seasoned developers 😃

@sovattha Do you use NRN as a boilerplate and keep it up-to-date with the NRN preset too? I know Samuel does.

[sovattha]

@Vadorequest My only remark is about this leading slash: I would have named the aliases @modules, @common instead of @/modules or @/common but I noticed from the documentation that it is already like this, so you can leave it as it is.
I used NRN as a boilerplate but I did not keep it up to date because I was happy with the snapshot I started from. Of course, I do not benefit from all the new features (+ I got out of sync with the up to date documentation) and I know that I leave some technical debts open.

[Vadorequest]

I believe the form @/common is recommended and becoming a standard, because people used to use common instead, which is confusing (am I loading an external module or local module? 🤔)

The @ doesn't work well either since we can have external packages starting with @ (@jest/core, etc.)

That's why the recommended way is @/. It makes sense and it's not confusing.

I wrote a small article about the process of migrating a Next.js app from relative paths to module path aliases: https://dev.to/vadorequest/migrating-next-js-jest-storybook-cypress-to-use-module-path-aliases-instead-of-relative-paths-d9a

[samuelcastro]

I just see this now, have you fixed the Jest issues you were having?

[Vadorequest]

I'm thinking about splitting the block (layout, pageContainer, footer, nav, head) components as some kind of "front-office" module.

My reasoning is that it'd be easier to add a "back-office" module with its own layout, footer, etc. components.

So far, I've considered all those files to be "default" (DefaultLayout, DefaultPageContainer, etc.) but maybe it'd make more sense to modularize them too. Also, it'd make simpler adding another "admin" or "backOffice" module that'd hide behind some kind of authentication, in the future.

I've done something similar with the "NRN examples" (renamed "demo") moved to a "demo" module, with its own components, and "demoSSG/demoSSR" used by "pages/[locale]/demo".

I'm wondering what you think of that, and also what'd be a good default naming. I'd like a name that means "publicly visible part of the site", unlike the "admin" which would be behind auth.

I'm thinking of stuff like:

• modules/siteFront
• modules/siteAdmin

[sovattha]

@samuelcastro @Vadorequest I just kept up with your conversation.

Indeed, I think core is a good name, instead of base.

I like the fact that all the custom code could live flat at the src level.

I reviewed the changes as best as I could, but 400 files is a challenge!

[Vadorequest]

Indeed, I think core is a good name, instead of base.

👍

I like the fact that all the custom code could live flat at the src level.

I also like the fact it's flexible enough for people to choose between /src or /src/modules at their convenience.
I think it might not be such a good design for huge projects with 20-ish modules living alongside /pages, but for small to middle-sized projects it would fit well. People can always move them to /src/modules if they wish so.

Would you keep NRN built-in modules in src/modules/core then? Do you like having them separated to locate your own code better?

I reviewed the changes as best as I could, but 400 files is a challenge!

Nobody can review that much, I'm only confident in this update because of CI/CD testing everything every time I pushed stuff, it'd be a nightmare without such automation.

[sovattha]

Would you keep NRN built-in modules in src/modules/core then? Do you like having them separated to locate your own code better?

Yes I would do so. I like to have at hand the folders/files that I am supposed to edit as a newbie along with the ability to use modules when the project reaches a critical size.

[Vadorequest]

Alright! I'll let things as they are now then, thank you both for your valuable feedback!

I'll merge today, once I've merged don't rush upgrading, I'll need to do the same work on the other preset, and then I'll also need to adapt my app and I'll share best practices and feedback experience about upgrading then.

[samuelcastro]

I just finished the review, everything looks great, I'm really excited about it, I was looking forward to see this change. 😄

The next big DX improvement I'd do is to include Prettier, I know we discussed this before, but without Prettier it's really hard to keep a consistent code style across different devs, but anyway it's a discussion for another time I guess. =)

Thanks for this hard work @Vadorequest.

PS: I also prefer core.

[Vadorequest]

@samuelcastro I migrated the gcms preset to the new folder structure. See PR #275

Everything looks good to me.

I found a few things I had missed, which were fixed directly in the above PR for gcms, they'll be fixed in a separate PR for the at preset.

[samuelcastro]

Depends, if you have one module per page (I kinda do) then it'd make more sense to have each gql query within the module.

That's what I do, each module has its own gql and all common queries I leave in my common/gql.

[Vadorequest]

Yeah I get it now, makes total sense.

[Vadorequest]

I'll do it now.

[Vadorequest]

Done :)

[samuelcastro]

Awesome!

[Vadorequest]

Okay, so I managed to merge all those changes into our private NRN fork.

We got about 100-150 conflicts, lots of things can be easily solved because they're about /demo. (about half of the conflicts)
The whole process took about 3h. I advise to do it if you can have 3h free of disturbance.

• Merge changes
• Fix conflicts (30mn)
• Once I fixed all conflicts, the merge operation broke and I had to git rm a particular file to continue the merge 🤷
• Fix all broken paths (1-2h) this was the most painful part, I'd suggest creating the directory and moving the core files in advance so that paths can be easily replaced instead of doing it manually after the merge, but I haven't tried that way, I went for the long and painful way of fixing all paths by hand
• Fix regressions (SSG/SSR, multiversalAppBootstrap and shared components (Nav, Footer, etc.) in particular (20mn)
• Move my own code to the right spot (10mn)
• Run codemod yarn codemod:module-path-aliases (do that only if you've committed everything) but the codemod broke because of some babel issue, it was working fine until now 🤷

Had about 500 modified files in total.

Good luck 😄