I've started running some LLM's on my computer locally as I want to see how they compare to chatGPT and copilot, and I've been pleasantly surprised. With that being said, the new stuff coming out like the below is getting crazy:
Machine Learning and Artificial Intelligence Thread
- Thread starter Johnnyvee
- Start date
Demos like that are often highly practiced and cherry picked. However this was still impressive, considering that kind of capability will continue to improve.
New programmer Devin.
The neural network model of conscious intelligence is plain wrong! But it will die a slow death, sort of like the exogenous cholesterol hypothesis of CVD. It (thinking machines) will never emerge from just adding up bits, even if you have all the classical computation that you could fit in the entire universe. ML can still be transformative in some areas like entertainment, but it will never be a thinking machine without the quantum component. A lot of quacks making money from selling books and hyping stuff now, it has similarities to the dotcom bubble.
This AI stuff is just hype. Silicon valey types desperetaly trying to be the next Jobs.
As far as programmers are concerned, they are safe for at least the next decade or two.
The current models developed very quickly to a level of being moderately useful to programmers, but they are still miles away from replacing them. From now on, it's going to take massive amounts of money and time to make tiny gains.
I predict in a few years, the hype will die down and every one will find their lost minds.
As far as programmers are concerned, they are safe for at least the next decade or two.
The current models developed very quickly to a level of being moderately useful to programmers, but they are still miles away from replacing them. From now on, it's going to take massive amounts of money and time to make tiny gains.
I predict in a few years, the hype will die down and every one will find their lost minds.
As a programmer who's been out of work for 8 months now, the market is very very tight and salaries are down 20%.
I think this is the normal ebb and flow of the job market. These lost jobs weren't replaced by AI, were they? Too many people entered the job market during the coof. It's going to swing upwards again in a year or two, I believe.
I also work in a technical field and it took me over 6 months to find my current position. The interview process is currently extremely burdensome. For one company offering mediocre pay, I had to sit through five regular interviews, a technical interview, and an exam. Only to be denied in the final round.
That being said, I don't think AI is to blame at this time because it can't yet do what we do. The job market is simply oversaturated. In large part, it's related to the performance of big tech companies. Last year's layoffs flooded it with job seekers, so even in other industries like insurance or manufacturing it's tough right now. People thought the covid boom of 2020-21 would last forever. I have a master's degree and several years of experience, so I can only imagine how hard it is for those less fortunate.
Just keep looking, you'll find something! You may have to compromise on something you don't like about the job like I did, but it sure beats unemployment. When you do get interviews, drop everything and study hard to pass them.
Now with the bigger picture of AI in the long run, I think people are going to have to start using it in their work. It can't replace humans yet because it doesn't have our intuition, but it will be able to do certain tasks very quickly including writing code blocks. And those who don't know how to implement it will fall behind.
I agree, but with the caveat that only a minority, (similar to what happened with covid) will find their lost minds.
People are going to give themselves up to it, even if it is mostly hype.
Compare the computer technology of when Windows 95 was released to today’s, a time span of three decades.
Those 30 years saw more technological change than from the foundation of the Roman Republic to the end of the Western Roman Empire, a period of a thousand years.
AI is basically at Windows 95 level. We could easily see the same 1995-2024 shift in technology from 2024-2039, i.e. half the time. Perhaps even a third of the time, so 2024-2034.
Let’s also not forget that even if you think AI is not actually “intelligent”, it does not need to be to replace hundreds of millions of jobs.
This is potentially not anywhere close to artisans losing their jobs as factories came into being during the Industrial Revolution and the next generations having new things to do.
Please feel free to critique and pick apart what I have just said.
I fall somewhere between the naysayers doubting AI and the outright pessimists, but closer to the pessimists.
Am I supposed to take that seriously? The deliberate delays that the 'AI's response is funny but them making the CGI work to make the thing look like it's Terminator is the icing on the cake.
I'm not denying AI or computer tech but that one only works on zoomers and boomers I guess.
There is no doubt that there is no AI at this very moment. The Casio calculator you had in the 80's is just as "inert" and without an "inner-life" as the ML tools of today. If AI comes about it will be from the quantum computation (QPU) field and not classical computation based machine learning algorithms. (although that can add a layer to it)
To make it as confusing as possible; what is currently dubbed "AI" in the media etc. is said to complete with QPU, and maybe "AI" will make QPU less relevant and so on...while in reality actual AI can only come from QPU's. I don't know enough about that field, but IBM seem confident that they can make significant progress in quantum computing and have a clear timeline. We also have to contend with the prospect of having central AI-QPU's if it ever comes about, since they require cooling which can't be realistically achieved in home/mobile devices, so it will only be via cloud and can hence be carefully controlled and censored.
To make it as confusing as possible; what is currently dubbed "AI" in the media etc. is said to complete with QPU, and maybe "AI" will make QPU less relevant and so on...while in reality actual AI can only come from QPU's. I don't know enough about that field, but IBM seem confident that they can make significant progress in quantum computing and have a clear timeline. We also have to contend with the prospect of having central AI-QPU's if it ever comes about, since they require cooling which can't be realistically achieved in home/mobile devices, so it will only be via cloud and can hence be carefully controlled and censored.
There is another way to look at this.
And that is that your Casio calculator is technically "conscious". It's just that consciousness is on a spectrum.
You could define consciousness as:
any ordered system in which information is transferred
This actually makes quite a bit of things conscious. But if you are talking about an inner life that is something only we have. And I'd argue it comes from above and not below (I'm referring to the quantum level as from below). The only theories I've heard tying quantum phenomena to our kind of consciousness are coming from materialistic perspectives and argumentation.
Moore's Law has already been disproven. You can't predict the future rate of development based on the past. There are initial gains to be made fast in every tech, but then you hit a plateau. Hardware progress has already slowed down significantly and will continue to slow down even more in the future, even if all the nerds of the world unite to prevent that. Unless another tech is found to make cpus like quantum, but I've been hearing about that for 25 years and it seems it was all hype just like AI.
Narekatsi
Oriental Orthodox
“Artificial Intelligence” is no intelligence at all. It is a very high speed matching and prediction algorithm. Developers, marketing gurus, and other geeks will come up with futuristic terms such as “Neural Networks”, “Training Datasets”, “Machine Learning”, “Mixture of Experts”, and so forth, to give the illusion that it’s a magical and sentient system.
Let me ask you this: Have you ever wondered why this is all done on a GPU and not a CPU? You could do it on a CPU, but it would be extremely slow... why is that? It all boils down to high speed floating point calculations, primarily used – in this specific case – to solve lots and lots of matrices.
CPUs have generally always dealt in integers (i.e. a negative or positive number without a decimal point). Calculations in decimals would – in the past – usually take place using additional CPU cycles, because the unit itself wasn’t directly handling the decimal numbers. This is why anyone who was a teenager interested in computers in the late 80s and early 90s, may remember the advent of FPU Co-Processors.
That was a big deal back then: All of a sudden, specialized software could offload floating point calculations (arithmetic with decimal numbers) to the co-processor (either on the motherboard or an expansion card). An FPU (Floating Point Unit) was not integrated into Intel consumer/workstation processors, until the 486 series. It was improved on from there onward with the Pentium, Pentium II, and so forth.
Why is all this important to understand? Because it brings us back to the original question: Why GPU and not CPU? The GPU is perfect for such calculations, because inherent to its nature are geometry and physics calculations.
Enter Nvidia and marketing. A CUDA “Core” (short for Compute Unified Device Architecture) is effectively a high speed, glorified FPU. So when you see Nvidia advertise that their GPU has 3072 CUDA cores, those are not cores at all – rather 3072 Floating Point Units. That doesn’t mean it’s not a big deal. It’s a very big deal, hardware-wise. What used to be a single FPU built into the Pentium processor in 1995, is now available 3072 times inside a consumer GPU.
There are also Nvidia "Tensor Cores" which, unlike CUDA cores, can do multiple operations at once, making them even better for solving matrices. At the end of the day, it's just one massive math problem being solved, over and over again.
Combine all of this with cheaper storage/memory, high speed transfer rates (between SSD + RAM + CPU + GPU), and the massive number of FPUs, and you have the perfect candidate for a prediction machine. All of this “AI” could’ve been possible in the early 90s, but when you take into account the hardware and storage required, it becomes apparent that even a “supercomputer” of that day couldn’t really pull off what you and I can in our high-end desktop system (some laptops, even).
What’s the point of all the above? That “AI” is a “smart” prediction mechanism. For example, in the case of the language models, it is simply predicting the next word (or few words) to follow. That is why we are also able to see the “streaming” output from some chat bots (i.e. as it types the response word by word). All the language model needs to do is mistakenly predict the incorrect word at some point, and if the mistake is material, it will build the rest of its response on that one error. Inherently, it is unable to “learn” or “understand” anything.
The same concept can then be applied to images/videos (diffusion techniques), and physics calculations. Once all three (language, visual, and physics) are combined and aligned together in a high speed fashion, it gives the illusion of intelligence, wowing the masses.
Can it be useful in speeding up certain things? Sure. But it's important to grasp that these systems are neither intelligent, nor truly learning in the traditional sense of the word. There is not and can never be any desires, emotions, or consciousness.
Let me ask you this: Have you ever wondered why this is all done on a GPU and not a CPU? You could do it on a CPU, but it would be extremely slow... why is that? It all boils down to high speed floating point calculations, primarily used – in this specific case – to solve lots and lots of matrices.
CPUs have generally always dealt in integers (i.e. a negative or positive number without a decimal point). Calculations in decimals would – in the past – usually take place using additional CPU cycles, because the unit itself wasn’t directly handling the decimal numbers. This is why anyone who was a teenager interested in computers in the late 80s and early 90s, may remember the advent of FPU Co-Processors.
That was a big deal back then: All of a sudden, specialized software could offload floating point calculations (arithmetic with decimal numbers) to the co-processor (either on the motherboard or an expansion card). An FPU (Floating Point Unit) was not integrated into Intel consumer/workstation processors, until the 486 series. It was improved on from there onward with the Pentium, Pentium II, and so forth.
Why is all this important to understand? Because it brings us back to the original question: Why GPU and not CPU? The GPU is perfect for such calculations, because inherent to its nature are geometry and physics calculations.
Enter Nvidia and marketing. A CUDA “Core” (short for Compute Unified Device Architecture) is effectively a high speed, glorified FPU. So when you see Nvidia advertise that their GPU has 3072 CUDA cores, those are not cores at all – rather 3072 Floating Point Units. That doesn’t mean it’s not a big deal. It’s a very big deal, hardware-wise. What used to be a single FPU built into the Pentium processor in 1995, is now available 3072 times inside a consumer GPU.
There are also Nvidia "Tensor Cores" which, unlike CUDA cores, can do multiple operations at once, making them even better for solving matrices. At the end of the day, it's just one massive math problem being solved, over and over again.
Combine all of this with cheaper storage/memory, high speed transfer rates (between SSD + RAM + CPU + GPU), and the massive number of FPUs, and you have the perfect candidate for a prediction machine. All of this “AI” could’ve been possible in the early 90s, but when you take into account the hardware and storage required, it becomes apparent that even a “supercomputer” of that day couldn’t really pull off what you and I can in our high-end desktop system (some laptops, even).
What’s the point of all the above? That “AI” is a “smart” prediction mechanism. For example, in the case of the language models, it is simply predicting the next word (or few words) to follow. That is why we are also able to see the “streaming” output from some chat bots (i.e. as it types the response word by word). All the language model needs to do is mistakenly predict the incorrect word at some point, and if the mistake is material, it will build the rest of its response on that one error. Inherently, it is unable to “learn” or “understand” anything.
The same concept can then be applied to images/videos (diffusion techniques), and physics calculations. Once all three (language, visual, and physics) are combined and aligned together in a high speed fashion, it gives the illusion of intelligence, wowing the masses.
Can it be useful in speeding up certain things? Sure. But it's important to grasp that these systems are neither intelligent, nor truly learning in the traditional sense of the word. There is not and can never be any desires, emotions, or consciousness.
Great post.
This writers captures the essence.
Just another load of fakery (after NASA etc) to fool the masses.
Indeed. These companies started calling these things AI because it was just an obviously good marketing move. Serious people don't think it's actually intelligent. A lot of normies do think that, but they also think AI can take over the world like in that movie they watched featuring the former governor or California, because they will frankly just believe anything.
That being said, this tech can absolutely change the world in a very extreme way. While it can only emulate/mimic intelligence, it's very good at it and that's more than enough for most use cases. People in this thread are underestimating it a lot imo.
As I've said many times before: "Remember, everything comes down to marketing."
