Mastering My Homelab: A Services Odyssey

I have the horsepower for an amazing homelab. Where do I gallop? That was my question as I decided on what I actually wanted to do.

First and foremost I love the IT space, but I am actively trying to transition my career into the Penetration Testing field of Cyber Security. I wanted to both self host secure environments for my services I was providing, but also insecure services that I could practice my pentest skills.

Side note here this will be a living document. As I create more detailed walkthroughs of installing services I will update this post with the links to the walkthrough if there is not a hyperlink yet I am working hard on getting that done but have wrote this in a way that feels like all the walkthroughs are complete.

Which Hypervisor

There are several choices when it comes to a hypervisor. I weighed the options and chose to go with Proxmox. I felt this gave me the best bang for the buck since it is free and fully supported. There are many guides how how to setup Proxmox, but here is mine

You Should Write That Down

The very first service I wanted to run was a wiki of some sort. A self hosted wiki would be a great place to document what I am doing with the homelab as well as be a repository of information when I got to the point of actually providing detailed write ups for those wanting to follow what I did. After installing a few different wikis I settled on Bookstacks. It came highly recommended from others and I liked the organization structure of it. You can find a detailed write up of the installation here.

It's Always DNS

My next goal was to set up whole house ad blocking. I truly believe that ads are getting way too intrusive. Have you ever looked something up on the internet on your phone just to have ads covering 65 percent of the screen? I wanted that gone, at least when I was at home. The most recommended whole house ad blocker is Pihole. Pihole was quick and easy to initially set up but there were some nuances to getting the best use out of it. In the future I plan to get a raspberry pi and run a backup instance of Pihole so if my main Server goes down the house won't lose internet. You can see my documentation of setting up Pihole HERE and the configuration of it HERE.

We Are On The World Wide Web

I wanted the experience of hosting a website from home. I am no web designer and have no aspirations to be one. So need a Content Management System (CMS) that was fast to setup and easy to use. I tried wordpress and some other CMS but I could not settle on one. I did not like wordpress as it felt to clunky to me. After some recommendations and trial and error I settled on Ghost CMS. You can find the detailed write up here.

What was that port number again?

At this point I had IP addresses, port numbers and various other details scrawled out on my notebooks. I wanted a dashboard that would be a central place that would allow me to set it up once and not rely on memory or a clunky bookmark folder in my web browser. The first dashboard I tried was Homer. It was super customizable and everything had to be done on the backend. While I loved the customizability of homer I didn't like having to go set up a new service on the backend of homer. I was spinning up and dropping a lot of services and I wanted it to feel more fluid. I replaced Homer with Heimdall. While Heimdall is not nearly as customizable as homer the easy of adding or deleting a tile on the front end was great. Here is my write up on setting up Heimdall.

Let's Git it

Services recommendations were coming from every direction at this point and a great one was a local git repository. I have a github but I thought it would be great to self host a Gitlab and only use github for things I want to share publicly. So that's what I did here.

Logs and Sweet Sweet Graphs.

With the number of services growing I wanted to check on the health and well being of everything I was running. That is were I set up 3 services Prometheus, InfluxDB and Grafana.

I started with Prometheus as it would be where all the logs would be ingested and cataloged to. Only main issue I found is having to go back through and install the node_exporter on every system. This was a simple install and the instructions can be found here.

Then to visualize the data I decided on Grafana. Prometheus and Grafana just go together With all the services providing data to Prometheus Grafana can poll the data and provide highly customizable dashboards that you could spend hours just configuring.

I also wanted to have pretty Grafana graphs with what my ProxMox server was doing. I could have just installed the Node_Exporter on ProxMox but decided to spin up a service that support was already built into ProxMox for which was InfluxDB. This did not take long and the walkthrough can be found here.

I Need a SIEM

I wanted to expose somethings to the internet. Before I did that I decided I should have some protections in place and the first thing I thought was a SIEM and an IPS/IDS system. Luckily my good friend Ben Heater had a great write up on installing Wazuh and was available when I inevitable messed it up. You can find his write up here. Mine can be found here.

Should I Be Monitoring That

I wanted a simple way to track and alert me if one of my services went down. For this I turned to UptimeKuma. A super light weight program that can monitor and alert me to when my services go down. Only issue is this is currently on my main server and if the main server goes down I have no oversight of if things are working. I plan to move this services off the main server to a raspberry pi but until it lives on my main server. The setup can be found here.

I'm Getting Tired of Doing Everything Several Times

When you start getting so many services running in your lab you eventually hit the point of frustration of going through each and every service to do updates or in my case when I installed Wazuh going through each and every server installing and configuring the agent. I turned to the most recommend way to automate tasks in a Linux heavy environment...Ansible. Honestly I wish I would have started with Ansible sooner it made my life much simpler. You can find my write up here and the bones of my playbooks here.