• 2 Posts
  • 15 Comments
Joined 1 year ago
cake
Cake day: June 15th, 2023

help-circle

  • If you signup to social media it will pester you for your email contacts, location and hobbies/interests.

    Building a signup wizard to use that information to select a instance would seemto be the best approach.

    The contacts would let you know what instance most of your friends are located (e.g. look up email addresses).

    Topic specific instance, can provide a hobby/interests selection section.

    Lastly the location would let you choose a country specific general instance.

    It would help push decentralisation but instead of providing choice your asking questions the user is used to being asked.




  • This advice isn’t grounded in reality.

    Management normally defines ways to track and judge itself, these are typically called Key Performance Indicators.

    KPI’s are normally things like contract value growth, new contracts signed, profit margin, etc…

    So if the project manager is meeting or exceeding their KPI’s and you walk up to their boss telling them the PM is failing as basic job functions, the boss won’t care.

    This is because the boss might have set the KPI’s or the boss might also be judged on them. In either situation its to the bosses advantage to ignore you.

    The boss will only care if there is a KPI you can demonstrate the PM failing to meet.

    Every person/group will have various incentives and motivations. To affect change you have to understand what they are.


  • A project manager has responsibility for delivery of a project but they typically lack domain specific knowledge. As a result they can’t directly deliver something, merely ask subject matter experts for advice and facilitate a team to deliver.

    Most PM’s cope with the stress of this position poorly.

    This cartoon is an example of micro management (a common coping mechanisim), the manager has involved themselves in the low level decisions because that gives a sense of control. If a technical team then tell them its a bad decison the team are effectively attacking their coping mechanisim.

    The solution isn’t to tell them their technical idea is terrible, when you’ve fallen down this rabbit hole you have to treat the PM as a stakeholder. They are someone you have to manage, so a common solution is to give them confidence there is a path to delivery, a way to track and understand it.


  • During the pandemic I had some unoccupied python graduates I wanted to teach data engineering to.

    Initially I had them implement REST wrappers around Apache OpenNLP and SpaCy and then compare the results of random data sets (project Gutenberg, sharepoint, etc…).

    I ended up stealing a grad data scientist because we couldn’t find a difference (while there was a difference in confidence, the actual matches were identical).

    SpaCy required 1vCPU and 12GiB of RAM to produce the same result as OpenNLP that was running on 0.5 vCPU and 4.5 GiB of RAM.

    2 grads were assigned a Spring Boot/Camel/OpenNLP stack and 2 a Spacy/Flask application. It took both groups 4 weeks to get a working result.

    The team slowly acquired lockdown staff so I introduced Minio/RabbitMQ/Nifi/Hadoop/Express/React and then different file types (not raw UTF-8, but what about doc, pdf, etc…) for NLP pipelines. They built a fairly complex NLP processing system with a data exploration UI.

    I figured I had a group to help me figure out Python best approach in the space, but Python limitations just lead to stuff like needing a Kubernetes volume to host data.

    Conversely none of the data scientists we acquired were willing to code in anything but Python.

    I tried arguing in my company of the time there was a huge unsolved bit of market there (e.g. MLOP’s)

    Alas unless you can show profit on the first customer no business would invest. Which is why I am trying to start a business.




  • Engineering is tradeoffs.

    A command shell is focused on file operations and starting/stopping applications. So it makes it easy to do those things.

    You can use scripting languages (e.g. Node.js/Python) to do everything bash does but they are for general purpose computing and so what and how you perform a task becomes more complicated.

    This is why its important to know multiple languages, since each one will make specific tasks easier and a community forms around them as a result.

    If I want to mess with the file system/configuration I will use Bash, if I want to build a website I will use Typescript, if I want to train a machine learning model I will use Python, if I am data engineering I will use Java, etc .




  • Just to add.

    Look at any hobby in your life and break out the money spent vs the enjoyment you got out of it.

    For example the Cinema costs me £10 and a film is 2 hours long, meaning my fun time costs £5 per hour.

    A £100 console would have to provide me 20 hours entertainment for it to be comparable to going to the cinema.

    These days any PS4 game will have 10-40 hours content, but buying them costs money. Popping on CEX website the most expensive PS4 games are £12. Assuning you only get 10 hours of fun from a game…

    The question you should ask yourself is are there 3 games on the PS4 you are interested in playing?



  • I think there is a focus on C/C++ to justify Pythons performance.

    There have been times when the performance of Node js/Python were part of the reasoning for choosing Java/Scala.

    Each time you are regaled with how Python is C/C++ underneath and so faster. Each time you have to ask if they will write C/C++ libraries to ensure the application performance meets our needs.

    Similarly any time you go near lambas you get a comment on how python lambdas are faster because its C running, where as Java has to start a virtual machine, etc…

    3 different companies samething



  • The big argument for mono repos is checking out multiple repositories is “hard” while checking out one repository is “easy” but…

    Service Oriented Architecture became a thing because monolithic code bases were often becoming spaghetti. I worked on a project where removing an option from a preference window (max map zoom), broke a message table (because the number of visible rows in a table (not its size in the UI) was linked to the max zoom you supplied to a map library, for no reason).

    Thus the idea you should wrap everything you do as a self contained service, with a known interface. The idea being you could write an entirely new implementation of a service, implement the interface and everything would work. Microservices are a continuation of this idea.

    Yet every node/python based mono repo I have seen will have python files directly imported filed from inside anouther component/service. Not simply common aretfacts but all sorts of random parts. Subverting the concept of micro service (and recreating the problem).

    Separate repositories block this because each repository will be built in isolation on a CI, flagging the link. This forces you to release each repository and pull things in as a dependency. Which encourages you to design code to support that.

    A common monorepo problem is to shove everything in a docker image and call it a day. Then if you need a class from one monorepo in anouther one, you don’t have an artefacts so lazy devs just copy/paste files between monorepos.

    Monorepos aren’t bad practice by themselves, they encourage bad practice. Separate repositories encourage good practice (literally the need to manage them separately drives it).