In the world of computing, I have observed an interesting trend over the last two decades. I thought it might be helpful to put it in writing and others might have other ideas on this.
Technology follows cycles and things eventually come full circle like a fly on a bicycle wheel.
Consider an example of technological growth in any area of computing.
- We build technology around resource limitations
- Once the resource limits are reached, we try to pool resources to solve the resource limitation problem.
- The consequence of centralization in Moore’s law is delayed and we get more power
- They have been pushing technology into devices again as more power becomes available and improvements are made.
- go back to point 1
Now this seems overtly simplistic, so I thought about some examples
Example: Artificial Intelligence / LLM / Gen AI
While the LLM world surprised everyone in 2022 with OpenAI pushing the boundaries with the unveiling of GPT. I have already made my thoughts on this subject clear. However, since we are talking about cycles, we again see similar cycles occurring in this area as well:
- First impression, this requires massive computing power and can only be run on the most powerful computers, the feeding frenzy has NVIDIA cards selling everywhere. For the first time after a while, GPUs were used for purposes other than mining cryptocurrencies.
- While the largest and most capable LLM models still require a lot of computing power, smaller models have started to emerge with a 7B or 13B parameter model capable of running and giving good quality results. With in-image quantification, system requirements slow down. Read here for local LLM setup efforts on legacy hardware. and I helped with that aspect.
- After the first wave of activity, the focus shifted to SLMs (small language models) to be able to make them work on smaller sizes so they could be run from a mobile device. Phi-2 from Microsoft, DistilBert from Google,
- Apple and Google have now confirmed that they are able to make language models work on mobile devices themselves and completely offline.
- Slowly we are moving towards the first complete change of moving the systems to the users and I am sure someone will once again suggest moving to a centralized setup for better control.
Example: voice translation
- tools with limited capabilities developed: natural language of the dragon, etc.
- centralized systems have taken over: siri, ok google, etc.
- voices against centralized power and increased CPU capacity on mobile devices have arisen
- voice translation service is now transferred to local systems
- more advanced in computer science with the aim of pushing/testing the current limits.
Apple m1 does this, mobile computer chips do the same.
Example: Web technologies
- Thin clients connected to the terminal’s central server do everything.
- The limitation hit and the PC became affordable computing transferred to the local.
- Core configurations and the need to perform complex calculations led to Web 1.0 server client
- Web 2.0 saw things come back to the client side as processors got better
- Chrome came along and started using more GPUs and CPUs and improving client-side JavaScript.
- Now the maximum limits are reached, organizations like Mighty are once again pushing IT to the cloud via a Mighty app or similar.
Example: Entertainment / Streaming
- Cinema and initial pictography were reserved for elites
- Little by little, technology became affordable, home cameras and video recorders for creation: cassettes, CDs, DVDs for storage.
- A centralization opportunity was spotted by a streaming service company due to the large amount of data
- Today the world realizes that due to centralization it is easier for streaming services to dictate their terms.
- People have now started going back to DVDs, or downloading and storing the data they like
Example: software development
- Languages like C/C++ focus on compiled code and primarily code written by developers.
- As development work progressed, people realized that modularization was useful.
- Modules were created and languages began to focus on using modules from other people.
- Module sourcing became a problem which was again solved via centralization of module sources (pip, rpm, deb, rubygems, etc.)
- People felt stuck trying to load modules into a centralized repository, so slowly that newer languages inherently allowed multiple sources.
- Languages like npm, go, etc. have a default configuration to allow multiple module sources.
- As languages like npm went to the extreme of modularization and allowed all modules to be downloaded to any location, the npm modules folder itself became a source of jokes.
- Modern languages like Golang or Rust focus on having modules, but the end result is a monolith containing all the necessary code in one place, precompiled.
Example: cryptocurrencies & web3.0
- The central authorities had too much power and people wanted things out of the hands of the central authorities. They envisioned bitcoin as an illegal or unauthorized, non-centrally controlled currency, in which the value of the coin would be decided by both parties to the transaction. A utopian dream come true: everyone has their own business and can decide on prices (remember the barter system).
- Bitcoin was OSS and, much like other OSS forks, began to build once divergent thinking came in directions.
- Litecoin, coin name to name a few, Dogecoin was created to make fun of the whole cryptocurrency craze.
- we even saw Bitcoin split in 10 second variations.
- Ethereum came and tried to do innovation. We also have forks.
- During this time, large numbers of people began to rally around the movement and work on the financial and business side of the equation.
- Trade needed some sort of governance, because rapid trade required people to trust each other. We can build trust through platforms.
- A trusted platform means huge transactions and lots of transactions mean a lot of money wasted on transaction fees. As prices rise, the transaction becomes prohibitive.
- Therefore, the system of off-chain exchanges and transactions started and crypto exchanges took center stage.
- From a fully decentralized concept, we have moved to fully centralized exchanges, which ultimately reward people for keeping money in the exchanges. I guess the situation will change again in a few years, when people keep things in their own hands rather than in exchanges.
That’s all for today. I will post more observations like this as I have more time. Subscribe if you like this style of writing and your thoughts will encourage me to share more.
Also share your own thoughts and observations on these trends in the comments below👇
An abridged version of this article was first published on linkedin. However, this is a revised and expanded version of the blog post.