Welcome to my reflection of my first 3 months weeehoooo

How it felt going into it

so this is the regular third year internship and at my school, it works a little bit different than other schools. Here it's like top 15 students or something like that and even then thats just for the chance to apply for internships, getting them is a whole different story.

The internal job board opened around the start of September and the initial search wasn't that bad. I had about 4 or 5 interviews and they mostly went well. The hiring process was relatively quick, they contacted me for an interview on September 23rd, did the interview on the 30th and then by October 3rd I got the offer. The company is a unique one for me since it's a mining company that really does not deal with software at all which is why me and one other guy were brought in. It's not a big company with only 44 employees, no swes btw, spread over like 4 countries with people constantly traveling to mining sites so it's a unique experience. We were brought in to create different dashboards and data analytics tools for the company and their clients which ended up being way more complicated than we thought it would be.

Quick Reality Check

As I said, me and another classmate being the only two software engineers is its own challenge and learning curve. We don't really have anyone to learn from or ask questions too regarding the software and especially the production level software that they need. for the first few weeks it honestly felt like going in there blind, not knowing what their data means (ive never looked at rock data), no clue how much we would have to scale this and what all the different use cases would be. Since this is still an engineering company, and a high quality one at that, most of my coworkers have masters or phds in mining and mineral engineering which helps with any questions I have regarding what this data means since i dont know if im pulling and manipulating things wrong or its their data that's the issue. You don't know the amount of times I saw a 700% value on Total Core Recovery which just doesn't make any sense.

The impostor syndrome lol. It was an interesting journey, from leading up to it, the first few weeks, and now being a few months in. I can start with how I felt before even starting it during the winter holidays. When they told us about the problems that they wanted to solve they said that they had an internal dashboard system that they made but it wasn't up to the type of standard that they wanted and overall it wasn't that good. Hearing that I was under the impression that we would be making a dashboard for the company and data may be messy but it's all in one source and is pretty structured. Once we started the internship we quickly learned that was not the case. When we started we learned that the dashboard we were replacing is used for all companies they consult for and their data can be very very different. For example a lot of the companies that they consult for do rock mining but every so often there is soil which is completely different and has to be accounted for. Another thing as we were learning about their data is that there aren't any table/column naming standards in their NoSQL database so even if two different companies are mining the same thing, how they name their fields can be completely different. Of course since no other software engineers, it was up to me and my classmate to figure these out, implement a standard that can work across all different mining types and the different data standards. A few of our constraints were having this be a single instance of the project and it also had to be configurable enough for multiple different mining companies while keeping its core functionality.

Something that really helped with the imposter syndrome or just not really understanding the scope of everything was my coworkers. I mentioned a lot of them have masters or PhDs in their field, and my CTO has a math PhD so whatever question I have I can ask and they can give me an answer I can understand. Things like data looking weird, how to calculate rock strength, etc. Actually just a few days ago we were starting a new project for a client and they wanted us to do some analysis on their data but we obviously have never done that. Turns out my company has written papers on calculating that with Hoek-Brown Criterion and doing that is running Bayesian analysis with Markov Chain Monte Carlo sampling and then for each sample get the Hoek-Brown curves. I love the shit like that and being able to see how other engineering fields work and how they take and modify the same tools that we use for other purposes. Another coworker, who just quit recently to move on to something else, also did something regarding AI during his PhD. Sitting beside him and talking about how machine learning can also be used to help make their jobs easier since a lot of the work they do at the mining sites is manual and takes a lot of time.

Mistakes were made

At work the ride was not always smooth. Actually it's probably been more un-smooth than smooth but it's part of it. Starting out we probably made every single rookie mistake imaginable. Not planning out what we would need for this, not having any design, rushing into coding. Yeah it was a mess. Luckily that only lasted like 2 weeks before we sorted all of that out. When we started it was a completely fresh project and we could do anything we wanted with it which we thought sounded amazing but we didn't realize what a trap starting fresh was. We had to be able to think about what the entire scope of the project was, what decisions should we make now that are going to make maintaining and changing this easier in the future. It was challenging for us since we don't really know anything about this industry and how they operate which is why I asked my coworkers a million questions. My obsidian vault is actually packed from reading papers about the industry and how data is dealt with in mining. I spent the time and did the full design in figma, bless figma make, we did real system design for what this web app will be, actually wrote down user stories and tasks. We essentially gave the project the proper respect it deserves and did actual project management and planning. After the first month things started being more organized and it was clear what we had to do. We were able to actually deliver on features and feel good about the type of progress we were making.

Building the product

We did a lot of research on how to implement something like this and what we landed on was a multi-tenant application which also drove the rest of our engineering decisions. Starting the work at the beginning once we had a bit of an idea was the hardest part , its a brand new canvas and the start of it will lay the foundation down for the remainder of the project. There are of course a lot of edge cases since there are so many different companies that are going to be using this with different variations of data (even the same dataset no joke would just randomly switch formats on a random date)so as we build we make our system more robust. We made a lot of mistakes in the start but after a month or so we found our groove and since we built this from the ground up we know our system and how it works. Since we opted for a multi-tenant application we render the proper dashboard based on a JSON config file. It's the foundation for the entire project and there was a lot to consider when making this: auth permissions, custom datasets, fields, logos, blob storage access, everything was meant to be customizable for the project. Every edge case had to be considered for whatever kind of project needs to be completed.

Probably the highest impact thing I did was MASSIVELY reduce the time that their photos took to load. Previously it was taking maybe 30 seconds to load all the photos and switching between the groups of photos was even longer. After looking around we found out they were loading over 6000 full quality images from azure which frankly is a lot. We did a lot of different things to fix this and we landed on storing CDNs in a postgres database and using a 2 layer cache system and maps for O(1) lookups for projects. Azure was nice in this use case for letting us use Event Functions to be able to generate thumbnail sized photos to be able to use over the web. Something else that took a large part of the time was creating the system for being able to add new clients and new projects into the app.

Actually shipping software

Since there are only two of us working on this dashboard all decisions are on us and we have to take responsibility for everything we do. At first it was a bit overwhelming buying all of these tools to use since for the most part I just use free / open source tools at home. Since its a global company and there is no tech team, my friend and I pretty much work on our own all week with no one to report to or micro-manage us which takes a lot of pressure off. We meet with our CTO and a senior rock engineer every Tuesday to talk about what we did last week and plan to do this week and other than that we can work autonomously for the rest of the time. I personally enjoy working like this because it's up to me to take full responsibility for my work and I do that well. Even though we don't have more formal meetings every week, if we have questions or want to give an update we can. It's a very startup culture place so I can really message anyone on teams. I think they appreciate that both me and my coworker can take initiative and do our work and we don't just sit and do nothing all day.

Once things started flowing properly and everyone was happy with the working prototype we decided that it's time to take ownership and show the entire company what we've done. Being able to be proud of the work we've done and being able to have something tangible to show was when there was a bit of a weight lifted off my shoulder of stress and "I'm not sure if this is good". Literally every single decision was made by us so being able to be proud of those decisions was a feel good moment. Based on everything, they felt great about the work we were doing. By April we pretty much had a full working prototype we could give to some clients to test out and give feedback and they want us to start on a new project right away. They actually enjoyed the work so much that they gave us a $7 raise which is lovely for wedding savings. That really showed me that what we're doing is valuable. Oh, and the other rock engineers here write python code for their testing and sometimes it's buggy or they need a new script for their specific task. I saw them complaining and I suggested Claude to some of them and they brought it up in a company meeting and a week later we had a Claude team account. Small things like that I think they really appreciate because a million times they've said "the mining industry is old school"

Personal Development

Starting this job also made me realize there are things I need to work on for myself both personal and professional things. One thing i think i'm not the best at is the entire corporate structure. I love programming and solving tough problems but constant updating and publicly showing my work was not my strongest thing since I would just let the work speak for its self. Anyways i quickly learned that that is in fact NOT how this shit works. Since then I started keeping a daily track of everything i did everyday, i even setup my workspace with this to make it easier. I started giving better weekly reviews of what tasks I have to do/got done which also leads to another skill that's maybe even more essential, speaking. My public speaking now is a million times better than it was before since I forced myself to give presentations in class but its still not where i would love it to be. We have a few meetings throughout the week and even if just giving a short "what i'm doing this week" I try to keep the same speaking standards so its just second nature like hooping. Tbh professionally and technically i learned how important it is to DO THE DOCUMENTATION. At the start of the project we were just coding away and not documenting anything except maybe what we did that day just in small bullet form. nothing about the code, how it works, what APIs do what. it was kind of a mess after another 2 months. I was auditing the code and I see two functions that do the same thing, comment above a function saying "nothing is using this just place holder" but when i delete the function the code breaks. That was poor documentation and we've learned since then to document everything even why we made decisions we did. Even reviewing the code is something I've had to get better at here. Gone are the times of not fully reviewing code, especially now that there's AI code in our codebase, which ironically makes it easier since we can use those tools to also review the code. Lastly, as much as it pains me to say... update the jira tasks, but don't accidentally assign your CTO a task 4 times lol

Where things are going

With the first dashboard coming to a close we're going to start moving on to other projects. The first being an external client dashboard which both me and my coworker feel much better about after having the experience of the first one. Working on their different dashboards and looking at all of their data has made us realize this data is really messy lmao. What we really want to work on is improving their automated QAQAC system to be able to catch these kinds of errors so even when they're doing their analysis they don't have random fake data. There are a bunch of other things that I want to make more efficient in their workflows so we have our work cut out for us. Maybe I'll go to a mining site one day. Who really knows.

Thanks for reading if you made it all the way here !

i swear next will be technical lol