I decided to create a full Web Developer Course that will cover all the technologies that you need to kickstart your Full Stack Developer Journey. No bullshit, everything practical and relevant will be covered.
We will delve into following topics –
Check out the first video of this series where we cover the introduction to World Wide Web and the Evolution of the World Wide Web. The code for the video (if any) can be found on GitHub.
In this article, let us learn about Running Geospatial Queries in MongoDB .
Geospatial Queries – Finding Places
Geospatial Queries are an interesting thing in MongoDB. You can fire queries not just for text, boolean, number, dates as condition but you can also create queries for locations like :
Find me all the restaurants within a radius of 2km.
Find me all the hospitals that near to this specific place
Running Geo Queries
Let us try to find all the places that are near my current location. So for this, let’s run this query. Make sure to use the latitude and longitude of your location before you fire the below query :
And here the $geoNear is the behind the scenes name of our $near query. For this we will need the geospatial index for this query to run. Not all the geospatial queries require the index but they all will somehow benefit from having such an index.
To add such an index we can use the createIndex method on the collection.
And if we repeat the same query, it should now succeed.
Now the question that must come in our mind is that how is near defined near, meaning that relative to what it is near. It does not make sense unless we restrict it.
We can also define $maxDistance which is a value present in metres here. We can also define the $minDistance which is also a value define in metres
Let us now write a query to find all places that are near to us in a certain radius distance.
Find out all places that are near to us in a certain radius distance
This answers our first question regarding which points are near to our current location. Now this area could either be in form of a sphere, polygon etc., let us say we want to find out which points are inside of that area ?
This is another typical question that we often encounter and in order to answer this let us add more points to our database. Let us add three more places :
Now let us run a query to find all the places that lie inside a certain area :
For finding such places, go to Google Maps : Inside the Google Maps Section
Go to Your Places tab.
Create a new map there.
Let us draw a polygon around our location.
$geoWithin will help us to find all the elements within a certain shape or certain object typically like a polygon. $geoWithin takes a document as a value and here we can add a geometry object which is just the GeoJSON Object
Store all the four coordinates inside the points p1, p2, p3, p4
Finding out if a user is inside a specific area. This can also be done using geospatial queries.
Let us see how we can find places within a certain area :
This is what we get as a result :
The $near method gives us the list of the places in the sorted order whereas the $geoWithin method will give us the list of the places in the unsorted order but we can sort them using the sort method on the records that we get back.
So this is it for this article. Thanks for reading.
If you enjoy my articles, consider following me on Twitter for more interesting stuff :
⚡Twitter : https://twitter.com/The_Nerdy_Dev
Don’t forget to leave a like if you loved the article. Also share it with your friends and colleagues.
Hey guys 👋🏻, In this article, let us understand about The NoSQL Database : Basics of CRUD in MongoDB. So in this article we will learn about how to retrieve to retrieve/access, delete, create and update our documents. We will also see how we can delete a document by making use of its id and all that jazz. We will be covering the basics about collections and documents, the basic data types and we will also see how we can perform these CRUD operations. So without a further ado, let’s get started.
Database, Collections and Documents
In the MongoDB world, you can have one or more databases. Each database can hold one or more collections. Now what is a collection ?
A collection is just like a table that we have in a relational database like SQL. In a collection that you and I create, we can have multiple documents. Something that we will discuss a bit later.
Now what do we mean by documents ?
A document is essentially a data piece that you store in the database.
You don’t need to worry about these much. When working with MongoDB database, collections and documents gets implicitly created for you when you start storing the data and work with them.
A bit later in this article we will also learn an explicit way of creating documents in your collection using which you can also configure them a bit further.
The Shell and the MongoDB drivers for different languages
You will use MongoDB drivers in the real world applications depending on the programming language you are using for writing the backend servers on.
Shell is the only approach that will work for every language.
MongoDB CRUD Operations
Shell is what gets connected to the local mongo server.
Creating Databases and Collections
To check what databases, we have on the server we can run the command :
Now to switch to a database, we can make use of the use command. If the database exists, then the use command in that case will make use of the existing one otherwise it will create the new one on the fly so basically on the execution of the command. So let us create a database and call it as flights
So to switch to the flights database, we can say :
We store collections in database. But in order to access a document we need to access that respective collection first in which the document resides, then only we will be able to access the document.
Understanding JSON data
Every document you enter gets a new unique id which is the feature of the MongoDB. You need to have a unique id for each and every document. We do not have to specify it on its own. Mongodb does this for us under the hood which is of type ObjectId which is another type supported by MongoDB. This id will allow you to sort the documents because it has some timestamp associated to it. This is how JSON data gets stored into our database, to be precise into the collection of our database.
To list all the databases that are there on the MongoDB server, we can run the command :
and this would give us something like this :
Now let us switch to the flights database. Now currently as you can see, the flights database does not exists. So to create it on the fly, we can make use of the command use flights and this will automatically generate a flights database for us.
As a result of the above command, we get this as output in our terminal :
switched to db flights
Let us insert a new flight into the flightData collection which gets implicitly created on demand. Now this new flight is a new document which gets inserted in the flightData collection of the flights database which we reference as db. So db here represents the current database to which we are connected to.
Now on running the above query, we get this as the output :
To display the array of flight documents in a pretty formatted manner.
First we reference the current active database db.
Then reference the collection whose all the documents we want to retrieve which in our case is flightData
The find() method retrieves all the documents in the current referenced collection.
So this is our query for same :
and this is the output for same :
Let us now do a comparison between JSON and BSON.
Comparing JSON and BSON
This is what JSON looks like :
JSON is what we insert or retrieve. Behind the scenes, MongoDB actually uses BSON data. This conversion is done by the drivers. This is simply done because BSON is more effective to store than the JSON data, faster and more efficient in terms of size and efficient storage. There are different types of data – numbers, binaries etc. which are stored in different ways behind the scenes
BSON stands for Binary JSON for storing data into the database.
Let us insert one more flight in our flightData collection.
This above insertOne does work because the collection can accept mixed documents since we know MongoDB is schemaless.
Two documents in the same collection don’t necessarily need to have the same schema
We may have the schema but it is not the must. The id which is autogenerated you just don’t have to use the autogenerated id you just have to ensure that you have a unique id but if you cannot ensure this you can assign the ids on your own.
Let us insert the same document which we inserted which had missing schema with the id of our choice. Make sure the id that you use is unique.
So here is the query for same. Please note here we are using an id of our choice i.e lmn-kpq:
After inserting the document into the flightData collection, we can find all the documents within the collection
Here the id is unique and it is the one that we used while creating the new document. It doesn’t have ObjectId type for the simple reason because it was not autogenerated by MongoDB it is the id that we gave to the document. In the next article, we will learn more about CRUD operations. See you next time !
Thanks for reading.
If you enjoy my articles, consider following me on Twitter for more interesting stuff :
Hey guys 👋🏻, In this article, let us understand about Security and Authentication in MongoDB. We will understand the security aspect from the perspective of developers and NOT for the database admins.
This article was first published on https://the-nerdy-dev.com/. Check the website for more interesting articles and tutorials on Web Development.
What are the most important parts for securing the mongodb database ?
For hardening the Mongo Environment and making sure that it is safe and cannot be tampered from outside, we need a security checklist.
Authentication and Authorization
The database that we will be using to store data and users will know the users and your code will have to authenticate as a database in order to get data, update data and do all kinds of stuffs with the data that you get back. It is the most important building block for securing your MongoDB environment.
Another important building block is the Transport Encryption
This means the data that you sent from your app to the server should be encrypted so that no MAN IN THE MIDDLE attack can compromise your credentials.
Encryption at Rest
This means that the data in the database also should be encrypted otherwise if someone somehow gets access to your database servers well they can then read plain text information easily.
So it is a must to store the data in the database in the encrypted format as well.
This is the pure server admin task and not the concern for the developer but Mongodb provides auditing to servers to see who did what and what actions occurred so that you can control and are aware of what is happening inside the database
Server & Network Config and Setup
Additionally the server on which you run database server (like physical machine that is running somewhere or the cloud provider like AWS) the instances that we book there the network that you are using for hosting your Mongo Server should also be secure.
Backups and Software Updates
As an owner of the database environment you should regularly take backup of your data. The softwares that you are running should be up to date.
Let us talk about the below three in great detail :
Authentication and Authorization
Encryption at Rest
Understanding the Role Based Access Control
Authentication and Authorization
Authentication is all about identification of the users in the database Coming to Authorization, Authorization is all about what these users may actually do in the database
MongoDB employs the Role Based Access Control System
Let us say we have the MongoDB server with three databases
Admin database which is the special database that exists out of the box
Authentication can be enabled in a very easy way and suddenly MongoDBs server only allows access to the authenticated users.
Let us considered a user like some data analyst/software developer who directly connects with our shell or say we have the app code that uses the driver to connect to the database. The analyst or developer is NOT a user of your application, not the user of the web application that you are building and we now need to login to the MongoDB Server. With our username and password, we can do that since authentication was enabled that means user needs to exist on the MongoDB server otherwise the login of the user will not be possible.
Now say we get loggedin but we have no rights to do anything…
The users in MongoDB are not just entities that are made up of username and passwords but they are also assigned some roles and these roles are basically the group of the privileges
A privilege is a combination of the resource and the action.
A resource would be something like the products collection in the Shop database and an action would be an insert() command for example to insert a product for example in our products collection
Actions are basically the task commands that we can do in our MongoDB database and resources define what resources we can access based on the access privileges that we are granted.
Typically multiple privileges are grouped into something called as the ROLES
That means a user has a role and that role includes all the privileges holding actions and resources that makes sense for this user.
This is the model that MongoDB uses since it is the most flexible model that Mongo has defined for its userbase. This allows us to create multiple owners where we can give every user exactly the rights that every user needs. We do not want to give every user all the rights because if we give all rights to any unauthorized person then they may do something malicious with our database which they were not designated to do.
Different types of database users
Admin – A real person who needs to be able to manage the database configuration and create users etc, create new databases, create new collections. The admin would need to be required to be work with data in the database. He does not need to be able to insert or fetch data.
Developer – A developer needs to be able to insert, delete, update or fetch data (all the CRUD operations that we discussed). The developer is not responsible for creating the users and manage the database configurations. This is not your job and the app code should not be able to do that.
Data Scientist – A data scientist needs to be able to fetch the data. He/she does not need to be able to create users, manage the database configuration or insert, edit, delete, update the data. His sole responsibility is to work with large amounts of data and derive valuable insights that are important for an organization. Working with large dataset along with strong analytic skills are a must for a data scientist.
CREATING A USER
Let us learn about creating and editing a user in MongoDB.
Users are created by a user with special permissions with the
createUser command. You then create the user with a username and the password. This user will have a couple of roles or atleast one role and each role will then contain a bunch of privileges. A user is created on the database. This does not limit the access of the user to that authentication database. But this is the database against which the user will have to authenticate. The exact rights the user has depends on the role that you have assigned to the user.
If we have the need we can also updateUser command, this means the admin can update the user that means for example we can use this to change the password
MongoDB ships with a bunch of built in roles to cover most of the use cases that you may require. You can also create your own roles but that is pure admin task.
We got a typical role for the users of the database
You also got typical admin roles like the
All database roles
Besides these roles we also have the cluster administration
Clusters are the concept where you have multiple MongoDB servers working together. So that you can have multiple machines running MongoDB servers and store your data which can then work and scale together. And managing this cluster of servers is ofcourse a meaningful task
root (the most powerful role)
root superuser can do everything
ASSIGNING ROLES TO USERS AND DATABASES
Run this command using the credentials of the created user
We can now authenticate in that user with the following command db.auth('appdev','dev')
This gives 1. This 1 signal indicates that this works.
Adding SSL Transport Encryption
We have our application and this could be Node, Django, PHP application that uses the MongoDB driver to communicate with MongoDB Server to store the data and ofcourse it is important that the data is encrypted whilst it is in transport so that someone who is spoofing our connection can not read our data. and MongoDB has everything for that built into it.
How we can secure our data whilst it is own its way from client to the server ?
To encrypt the data whilst it is in transport, MongoDB uses SSL or actually TLS for encryption and uses public private key pair to decrypt this information on the server and to prove to the server to prove that who we are. It is secure way of encrypting our data and decrypting our data on the server.
While its own its way it is consistently updated.
ENCRYPTION AT REST
The data which is stored on our MongoDB Server in a file this ofcourse might also be encrypted so that we can decrypt there two different things
The overall storage (feature built in for enterprise versions)
The thing as a developer you can do is to encrypt certain values in your code. For example, if you are storing the user password you should hash that password and not store the plain text you can go so far for all data you always have a way for encrypting that.
So you can encrypt both your data as well as the overall file storage to achieve max security that is possible
Here are some of the important links that you must reference to know more about Security and Authentication in MongoDB:
In this article, let us learn about how we can render dynamic content to the client (which is not static). Let us first touch briefly on what a template engine is because rendering dynamic content is closely related to what a template engine will help us in achieving.
Template Engines – a brief overview
A template engine is software designed to combine templates with a data model to produce multiple pages that share the same look throughout the site. These are the views in an MVC project.
So this helps us in putting dynamic content on our HTML pages.
We got a HTML like template (technically not HTML) which is typically a file that contains a lot of HTML like content in it but with some placeholders plugged into it and then we have a server (a Node.js Express server or any other server) serving the Node/Express content and then you have a templating engine. that replaces placeholders with HTML content but that content is generated on the server dynamically on the fly taking that dynamic content into account.
Some of the common template engines :
Dynamic content to be rendered in the above example is name. The value of the name variable will be rendered as an HTML page’s paragraph with value of name getting rendered on the fly.
PUG uses minimal HTML and custom template language.
Handlebars uses normal HTML and custom template language
The side effect of Handlebars template compared to Pug is that we cannot run any logic in the Handlebars template we need to pass a variable that has a value true or false(i.e logic is performed outside and passed into hbs template) and then we can render dynamic content.
So this is it for this article. Thanks for reading. Don’t forget to leave a like if you loved the article. Also share it with your friends and colleagues.