We have a web app in which data is loaded from mysql database.
Active users make a request every second.
http.Handler on each request retrieves same data from db.
I want to save last 100(or 200) in global app memory and first will try to check this by key in cookies. If user not in cache then make request to db and add data to cache.
The same approach can be used with other frequently requested data.
I want to know what gophers think about this approach.
I would be grateful for your advice and useful links.
Related
I found this quote on the Vercel website:
When using Next.js, Vercel optimizes Serverless Function output for server-rendered pages and API Routes. All functions will be created inside the same bundle (or in multiple chunks if greater than 50mb). This helps reduce cold starts since the bundled function is more likely warm
But is this also true for getServerSideProps?
I have a small project with an API and another page that loads the data with getServerSideProps. When the first API is done. I would except the next page with the getServerSideProps would be fast, but that also seems to have a cold boot.
The second time everything is fast.
Based on the two following comments from Vercel in related GitHub discussions:
One thing to note — in Vercel (a recent update), Next.js SSR pages and
pages/api routes get bundled into two separate Lambdas (λ), so you
should not have to worry about the Serverless Function limit.
Source: https://github.com/vercel/vercel/discussions/5093#discussioncomment-57628
API Routes will all be bundled, each SSR page will also be bundled -
both of these will be creating another function should they exceed
maximum capacity (50MB) although this is handled a little before the
limit to avoid it being too close.
Source: https://github.com/vercel/vercel/discussions/5458#discussioncomment-614662
My understanding is that in your scenario the API route (or any other API routes you may have) will be bundled into one function, and getServerSideProps into another function. If either exceeds the 50MB size limit then an additional function would be created.
What you're experiencing seems to be expected. The API route and getServerSideProps would be a different functions, thus having separate cold boots.
You can also use caching headers inside getServerSideProps and API Routes for dynamic responses. For example, using stale-while-revalidate.
if you use Vercel to host your frontend, with one line of code you can take advantage of the stale-while-revalidate cache strategy which has very similar behaviour to getStaticProps when using revalidate.
// This value is considered fresh for ten seconds (s-maxage=10).
// If a request is repeated within the next 10 seconds, the previously
// cached value will still be fresh. If the request is repeated before 59 seconds,
// the cached value will be stale but still render (stale-while-revalidate=59).
//
// In the background, a revalidation request will be made to populate the cache
// with a fresh value. If you refresh the page, you will see the new value.
export async function getServerSideProps({ req, res }) {
res.setHeader(
'Cache-Control',
'public, s-maxage=10, stale-while-revalidate=59'
)
return {
props: {},
}
}
Check the docs: https://nextjs.org/docs/going-to-production#caching
But it is important to know when to use getServerSideProps or api urls in next.js
When to use getServerSideProps
The getServerSideProps function fetches data each time a user requests the page. It will fetch the data before sending the page to the client If the client makes a subsequent request, the data will be fetched again. In these cases it is helpful to use:
SEO is important for the page
The content that is fetched is important and updates frequently
When we want to fetch data that relates to the user's cookies/activity and is consequently not possible to cache. Websites with user authentication features end up relying on getServerSideProps a lot to make sure users are properly signed in before fulfilling a request.
If the page is not SEO critical, for example, if you need to fetch the current users for the admin, you do not need to use getServerSideProps. However, if you are fetching the shopping items for your home page, it is good practice to use getServerSideProps.
When to use api functions
It is important to understand that some websites that we are building do not just need Html pages that are being served back to the users upon requests. You might for example have a feature on your website that allows users to submit feedback or signup for a newsletter. So when a user clicks on such a button to for example signup for a newsletter, think about what happens behind the scenes when we signup for newsletter.
We need to send data to some server to store that entered newsletter email addressd in some database, and that request being sent is not about fetching a site, it is about storing data. We do not want to get a Html page, Instead we want to send that user entered data to some database. that is why we use API's for. there are different types of api but the main idea is same. we have a server that exposes some certain urls. Those urls are not just sending html data, but they are about accepting some data and sending back responses with any kind of data.
So Api routes are special kind of Urls, which you can add to your next.js application, which are not about getting a standard browser request and sending back a prerendered html page, but which are about getting data, using data maybe storing data in some database and sending back data in any form of your choice.
Here is the situation, I have setup 2 codeigniter installation.
One will be a client and one will be an api. Further improvement of this will be
The client will no longer be made from CI, since I wasn't using it's functionality. I just wanted to start out from a mvc framework right on.
My question would be where should I be storing sessions? during logins.
Below is how I did it, but I think I did it wrong.
I created a Login from the client. This one sends the login credentials to the api and then validated these information sent by the client and will return a message/response whethere the login credentials were valid or not.
If the login details were valid, the api will set a session in it's controller like this
if(true) {
$this->session->set_userdata($array);
}
This is in the login_controller I created. Is this the proper way of setting sessions for a client of a api?
You've got the concept right. You only want to set session userdata upon verifying the user supplied valid credentials.
That said, make sure you're using encrypted cookies and, if you're handling sensitive data, store your session data in the database. Storing it in the database causes some odd quirks with how sessions work in CodeIgniter (mainly with flashdata), but the added benefit of positive identification might potentially be worth it.
By storing the session data in the database, you can more positively verify a user is who they claim to be (in terms of the session ID, etc). The reason is because the session data is stored only in the database, and not in the session cookie (which only holds session ID and some other info). That way, even if someone does manage to decrypt the cookie, they can't modify their userdata to pretend to be someone else, like you might be able to with the cookies only method.
I did a little bit of Web Programming here and there but I never quite understood what's meant by the word Session.
I've googled a bit here and there, read the Wikipedia article, but could never quite grasp the meaning of it.
So, what's a Session?
Session is a way of persisting your information across multiple pages and requests. When you visit the login page of any site and you provide your username and password, you won't need to provide them again on subsequent pages.
This is done by attaching a session id, unique to your request, and is sent back and forth as you navigate pages.
Session Id could be stored in cookies (file on your system), in the URL as part of query string or in the database
A session is a place for storing data for a particular visitor of your site.
You can store data there that is also available on the next page request from that visitor. If some data is stored 'in the session', it means that the data is stored somewhere (possibly in the database of the server or in files) which the server can then use to construct the web page.
The visitor will receive a temporary cookie which contains a session id, an identifier which is used to associate that visitor with the session data that is stored on the web server.
The session id is sent to the server with each request and the server can lookup the stored session data (which can then be used to construct the web page).
It's the concept of keeping state around over an inherently stateless protocol like HTTP.
If you want to keep track of a logged-in user, for example, and maybe some data associated with that user, you could send that data between the server and the client each time, which of course would be terribly insecure. Or you could keep it in a session store on the server, for example a file or a database, and just exchange an identifier for the storage location between client and server. That's usually done via cookies these days, but could also be a parameter in the URL.
To make it simple:
If you first visit the site, the server gives the client an identifier. With this the server can identify a client across several request from the client to the server. The identifier is deleted after a preset time.
The combination of this identifier and the timeframe the identifier is valid, is called session.
Hope that helps. :-)
Session: An interaction between user & server, which has an ID associated with it. So that server can pin-point & serve the users according to their requests. Cookies are basically used for storing the session information because by default HTTP is state-less.
How can I increase the security of my sessions?
$this->session->userdata('userid')
I've been throwing this little bad boy around for my ajax calls. Some cases I haven't. Then I was like, is this really secure using id from the DOM? what if the DOM is changed to hack user accounts data? So then I was like I guess anytime a user is doing something relating to their id, only sessions should be referenced. Am I right?
Referenced like so:
$this->some_model->do_data_stuff($dataId, $this->session->userdata('userid'));
Then I read this:
While the session data array stored in the user's cookie contains a
Session ID, unless you store session data in a database there is no
way to validate it. For some applications that require little or no
security, session ID validation may not be needed, but if your
application requires security, validation is mandatory. Otherwise, an
old session could be restored by a user modifying their cookies.
http://codeigniter.com/user_guide/libraries/sessions.html
I'm not going to be storing financial data but I don't want any data on my site corrupted ever. Does SO use session validation? How much overhead will this validation cost? How would a session be hacked? What are some things to look out for with session security?
Using CodeIgniter sessions with database is going to be fairly secure. You just don't have to trust the input that the user gives. Even if you are using AJAX, the CodeIgniter session will work just like any standard call, so the same security goes on.
What happens with the CodeIgniter session is that the server stores the cookie, and every time the user does an action that would change the content of the cookie, it is first compared to the previous cookie.
If the user changes the content of the session cookie in the browser, CodeIgniter will notice on the next server call, and create a new session for the user, basically logging him out.
CodeIgniter doesn't really need the data stored in the cookie in the user's browser, and as long as you're using
$this->session->userdata('userid');
you're going to get trusted server-side data. The user can't change that. Furthermore, the cookie can be encrypted, and you should have it encrypted. Just look in config.php of CodeIgniter.
There are several other protections around the session data: the short refresh timeout (usually 300 seconds), it checks if the IP changed, and if the browser changed. In other words, in the worst case scenario, the only way to spoof the session data is by having the same version of the browser, having the same IP, getting direct access to the computer to copy/paste the cookie, and getting this done within 5 minutes.
So, watch out for the guy sitting beside you!
I want to use cache in this way for the application I have.
in first request data come from database and passing them to view from controller and storing them in cache.
in next request I want to get the data from cache and putted them in page and asynchronously get the data from database again and check if something new come then update the cache so.
because in last request data come from cache it's goes faster and update the cache for future request.
How I can implement this thing in my application [MVC 3]?