The columns on either side of the main body of the site are called sidebars. There is typically a right and left sidebar, but more often they are calling them first and second since you may choose to have them both on one side as opposed to one on each side.
All of the information in the two sidebars are "blocks." You can access the blocks under Structure ยป Blocks
You can control where the blocks will show. Most often you're going to use the left or right sidebars.
You can also control in what order the blocks show up in based on their weight -- smaller items float to the top.
You can use the blocks already created for you (such as search, log-in, etc.), or you can create ones of your own.
Choosing when/where the blocks show up
You can also control when/where the blocks will show up. This is handy for when you want an item to show up only on specific pages, such as a table of contents for your monthly newsletter.
Click "configure" next to the block you want to edit, Scroll down. Towards the bottom is an area called "Visibility Settings" where you can choose when and where the block shows up.
Save the block.