Divar-starter-kit: How do we do SSR in Divar.ir
We recently decided to open-source our React.js SSR boilerplate, divar-starter-kit in Divar.ir web chapter.
In this article, besides introducing divar-starter-kit and how it works, I tell you the story and journey behind it. I tried to explain the main idea for non-familiar readers in easy words and also more technical stuff for those who are more interested.
Divar-starter-kit
Divar-starter-kit is a React.js SSR-ready boilerplate using Razzle by Divar. It also contains tool-chains for quest start and offers file-structures that we often use to develop our front-end projects. The project mainly focused on Server-Side-Rendering but also helps you to have a quick-start and directly jump to developing.
Features:
- React and Razzle features.
- Next.js like SSR features without file system routing.
- Redux ready (CSR and SSR).
- Using React-router-config.
- Recommended Directory Layout.
- HTTP service included (using Axios).
Easily clone the project from this repo and fire it up!
git clone https://github.com/divar-ir/divar-starter-kit.git
cd divar-starter-kit
yarn
yarn startThen open http://localhost:3000/ to see your app. Your console should look like this:
Divar website from past to present
Divar (means wall in Farsi/Persian) is the largest horizontal classifieds in Iran with over 35 million users and more than 500 thousand daily ads. Many of our users use web application and creating a great user experience always is one of our goals.
Why Server Side Rendering is important for web businesses?
Divar web client was a Single-Page-Application (after migrating from the first version — server template rendering with Django). Using SPAs create a grateful UX and DX in most cases but any improvements come with expenses.
SPAs (in a simple way) are created from a JavaScript file (source-code bundle), CSS files, and an index.html entry source. JavaScript main bundle download and runs in user’s browser. After that our SPA starts to mount/render UI and DOM nodes and fire AJAX requests. It takes more time (downloading bundle, running code, create DOM nodes, request data, and finally renders data) than previous version (server template rendering) for users to see website (UI) and content (like posts lists) for the first time. It also affects SEO ranks because crawlers won't wait to see your website loading and then show content (or they may not running JS probably).
There are two main approaches to this problem:
- Pre-Rendering / Static Site Generating: build your all pages as static HTML files and serve. You can deploy them on a CDN and have a great site speed without any server costs. It's a great approach for low refresh rate content websites (like news sites — about 10 builds a day for each post). Not a good option for Divar.ir — about 10-15 new posts posted in every second).
- Server-Side Rendering: Request data and render the page that the user wants on the first website request in server. Users and crawlers see meaningful content while the bundle is downloading and mounting.
Our “previous” approach for doing SSR
We first did SSR in React without any external tool for THE-WALL (web client internal code name) many years ago.
In that approach, an Express.js server and many middlewares handle managing requests, fetching data, initiating redux store, managing routes and chunks, and at the end, rending HTML using React renderToString.
A view is a component that renders by the router on a specific URL.
In our implementation, there were syncServerSideInitial (for doing sync jobs) and asyncServerSideInitial (for async jobs) static methods on each view for doing SSR. These methods called on server-side for fetching data or dispatching redux store actions.
For every request on server-side (after finding matched chunk with request URL), views that wrapped in SSR HOC extracted and their static methods (syncServerSideInitial and asyncServerSideInitial) called.
Also on each request an empty redux store created to be used in async SSR methods and all Async SSR methods called by Promise.all().
In the end, whole app wrapped by store provider with initial data rendered to string, and inject in HTML-template (HBS).
In client-side, like server-side, the app renders inside store provider with SSR initial data using renderRoutes on the route.
Problems with the “previous” approach
- First, having various methods for SSR (async and sync) for doing SSR added uneccessery complexity to views. It also couples views to SSR implementation.
- Second, if a view needs SSR, we should use store to manage loading state and prevent repeating data fetching. However, the use of store has its own logic and reasons. In addition to unrelated use of store for our case, it added a bunch of code like reducers, actions, action-types, and registering reducer and connecting to store.
Revision to SSR approch
In 2019 and in order to start developing our new brand site: karnameh.com, we did research and review on our SSR approach.
In Divar culture, we found answers by benchmarking available solutions based on our needs and create a detailed design document about decisions and answers.
A design document includes describing the problem and needs, results of benchmarking solutions, and describing the final result in a technical way. It helps to share knowledge and discuss it in the web chapter and save the results for future returns.
We wrote a design document for revisioning our SSR approach.
Our needs were:
- Typical server-side rendering.
- No structural and technical limitations and the possibility of using different libraries (especially react-router)
- Ability to use Redux in server-side and client-side.
- Access server-side data in client-side without using store.
- A better developer experience.
After benchmarking platforms, libraries, and boilerplates available at that time and documenting the results, we finally came up with a few key candidates:
- Next.js: This framework seemed to be a good option due to good development and support, large community, and easy and fast startup, but the impossibility of creating nested routes caused the components to repeat. Also, its routing system is opinionated, which imposes serious restrictions on the project structure and technical cost of migration from the current routing structure (react-router-config).
- After.js: The use of this framework was close to our technical and product needs, but the lack of proper nesting routing at that time and of course, the lack of maintenance were the main reasons for not choosing this option.
- Final Result — Implementing SSR by ourselves:
Once again, considering the needs of the products and the fact that we intended to keep the platform of front-end projects same and help reduce technical costs and easy transfer between projects, we decided to develop SSR ourselves once again and upgrade the codebase to make exactly what we need. We knew that using previous SSR development experience reduces technical and time costs. Of course, we also got help from Razzle.
Using divar-starter-kit
In general, to use existing experiences and prevent duplication, we set the starting point based on the method in the THE-WALL project. The difference is that in addition to adding features such as Hot Module Replacement to the server-side during development, it reduced a significant amount of code and complexity associated with webpack configurations and development tools due to the use of Razzle. While the capabilities of the previous method (specifically sending redux store data from server to client) were still supported.
In this method, which is similar to the method used in Next.js and used in divar-starter-kit, to SSR a view, we need to define a static method called serverSideInitial that receives the necessary inputs and can do sync or async operations. If data from the serverSideInitial function is required in the view, we must return the required data in this function. This data can even be a Promise.
Finally, to receive the data returned from the serverSideInitial method, we need to wrap the view inside a HOC called withSSRData. This allows the view to receive the data into a prop called initialSSRData, which is the value returned by the serverSideInitial or the resolved value of the Promise inside.
Divar-starter-kit in depth
We used a react-context to store the received data from the serverSideInitial functions, which uses in server-side and client-side as provider and in withSSRData as consumer.
In SSR, a new instance of this context is created for each request. Due to the need to access the context withSSRData, after each context is created on the server, its value is stored to be accessed by importing the getContext function. In the client, an instance of context is created at the beginning of the program.
The data structure within this context is as follows:
- A data property that contains all the data related to serverSideInitial (with the structure described below).
- A function called clearDataByKey is used to clear data for a view after use.
On the server-side, it iterates on the branch of the views that matched with the request URL to extract an array of the following items in shape on an object. It should be noted that the order of the array is in accordance with the branch structure:
- ServerSideInitial method of each view
- Determine whether is a view requires the returned data of that method. Detection of this point is done by checking if the view wrapped inside withSSRData HOC.
- Path related to the view. Which will then be used as the unique data storage key of that view.
All extracted methods are called a Promise.all . As the Promise.all behavior, both return-value values of functions (whether Promise or not) is supported. Promise.all results are placed in an Object (called preloadedInitialData) for each item with the key of each view’s path, if that the view requires the data.
After that, the app is wrapped inside the context provider with the value of preloadedInitialData and the provider of the store and converted to a string by renderToString.
Finally, the string is placed in the HTML-template and sent to the client as a response. Also the value of preloadedInitialData and state is added to a script tag within specific properties on the window.
On the client-side, like server-side, the app is wrapped inside context provider with preloadedInitialData and store data. And the initial value of both items is received from the same special properties on the window that are filled by the server.
withSSRData HOC is actually the initialSSRData consumer, which uses the getContext function on both the SSR and CSR sides to access the appropriate instance of context.
withSSRData by using withRouter, receives the path key for the current page, and send current view data with path key to the wrapped component inside a prop called initialSSRData.
The component clears the data in the context with the help of the path key and the clearDataByKey function in context on componentWillUnmount to does not cause any problems in receiving incorrect data in other instances of this component.
This method allows us to customize or improve caching in the future, if needed, with a slight change. Also, placing a static member (named HAS_PRELOADED_DATA with a value of true) on the component it creates helps identify the initialSSRData needs of the page.
Divar-starter-kit on production
We have been using the divar-starter-kit directly on our projects like Karnameh and Suke for a year.
The revision of the SSR approach has had a direct impact on the ease and speed of developing our products. The development of divar-starter-kit, in addition to solving our problems and needs, has helped to reduce the size of source code, project tool-chain, and project start-up time, and have a cleaner codebase.
Feel free to send a PR in divar-starter-kit repo or comment below.
About writer
I’m MohammadReza Iranmanesh, Front-end Engineer in Divar. Get in touch with me on Twitter or Telegram, about this article, technical stuff or Iran's tech and start-up community, and everything else.
