Honestly, Tensorless was not supposed to become this big.

I was learning PyTorch.

I wanted to understand how neural networks and model training actually worked, so before starting Tensorless, I built two AI models with PyTorch.

Those two projects were mainly for learning. I wanted to understand the basics by actually building things instead of only reading tutorials.

And after doing that, I started thinking about something pretty simple:

Why does training a model have to involve so much setup every time?

If I already have the training data, why can't I just give it to a framework and let it figure out most of the boring stuff?

That was the beginning of Tensorless.

First, I Built Two Models With PyTorch

Before Tensorless existed, I spent time learning PyTorch by actually building two AI models.

That gave me a basic understanding of how the whole process worked:

• Preparing data
• Creating a model
• Training
• Loss
• Optimizers
• Batching
• Saving models
• Loading models
• Inference
• And all the configuration around training

Building those two models was important because I wasn't starting Tensorless completely blind.

I had already experienced the process myself.

And after doing it twice, I started noticing how much of the setup was repetitive.

Sometimes I don't want to spend a huge amount of time configuring everything.

Sometimes I just want to get a result quickly.

So I started thinking:

What if most of this could just happen automatically?

The Original Idea Was Actually Pretty Simple

My initial goal wasn't to create another huge machine-learning framework.

I wanted something more like:

Give Tensorless your training data
↓
Tensorless figures out the setup
↓
Train
↓
Get your model

The idea was basically:

Put data in → go.

If you're a beginner, you shouldn't need to understand every training parameter before you can train a model.

But if you're an advanced programmer, you should still be able to configure everything yourself.

So the philosophy became:

Simple by default. Configurable when you need it.

The First Tensorless Version Used PyTorch

The first version of Tensorless actually used PyTorch underneath.

I wasn't trying to reinvent neural networks at that point.

I was trying to automate the training process.

The configuration was mostly determined automatically using fixed conditions in the code.

So instead of manually deciding every parameter, Tensorless would make those decisions for you.

That was the first version.

And honestly, it worked surprisingly well for what I wanted.

But then I ran into another problem.

I Wanted It To Be Small

One of the reasons I started thinking differently about the architecture was the amount of stuff involved in a PyTorch-based workflow.

When I was experimenting with the project, the dependencies and modules could become 500+ MB.

And I started thinking:

"I'm trying to make training simpler and lightweight, but I'm bringing in a huge stack just to do it."

I didn't want Tensorless to become a massive package full of components that users might never need.

I wanted the package to contain the things actually required for its own workflow.

So I decided to go much further.

Then I Made A Slightly Insane Decision

I started building more of the system myself.

Instead of keeping Tensorless as a PyTorch wrapper forever, I started working toward my own neural/training system.

That meant getting into things I originally hadn't planned to touch.

I started working on:

Neural network components
Training
CUDA optimization
Data optimization
Data pipelines
TPU support
Automatic configuration
Tokenization
BPE
Model saving/loading
And other training infrastructure

At this point, Tensorless had changed completely.

It wasn't just:

"PyTorch, but with easier configuration."

It was becoming an attempt to build a lightweight training system around the workflow I wanted.

Then I Needed Something Real To Test It On

So I built a small chatbot.

A cat chatbot.

I called it CatTongue. 🐈

The model was trained on CPU, and with the PyTorch-based version of my training system, I was getting results that were honestly pretty fun.

For example:

You: "Do you trust me?"
Cat: "only if you walk toward the kitchen. 😼🐾"

You: "Who is your enemy?"
Cat: "empty boxes. 📦😾 they are useful for exactly zero seconds."

You: "Who is your favorite human?"
Cat: "are you holding food? 🤨🍗"

You: "Do you have a best friend?"
Cat: "the vacuum cleaner. 😾🧹 we have unfinished business."

It wasn't perfect.

But it was understandable.

The model was actually producing responses that made sense.

And I thought:

Okay. This is working.

Then I Changed The Training System Again

I continued developing my own training implementation.

And then I trained again.

This time...

OH MY GOD.

The results were terrible. 😭

I started getting things like:

You: Hello
Cat: yes. � I understand the ancient languagnacke.

You: do you want food?
Cat: food. �� � I have interrupt the sound forctinking.

You: fish?
Cat: food. � � � suspicious.

You: who is your enemy?
Cat: yes. � I amain but jobs.

You: meow
Cat: okay. � I superiore your fish.

You: kitchen
Cat: fish. � They are you mrrp.

And my reaction was basically:

FAAAAAAAAAAAAAAAAAAAAH.

😂

There were recognizable words.

There were sometimes recognizable meanings.

But the language wasn't behaving properly.

And this was actually one of the most important moments in the project.

Running a Neural Network Isn't The Same As Making It Learn Properly

I had already learned how to make the system train.

I had a model.

I had data.

I had a training loop.

I had the infrastructure.

But getting something to actually learn useful language representations is a completely different problem.

That experiment showed me that my implementation still had a lot to improve.

And that's where I started understanding the difference between:

"The code runs."

and

"The model works well."

Those are VERY different things.

But The API Was Becoming What I Wanted

Even while the underlying system still needed work, the interface was becoming extremely simple.

For example, pretraining and fine-tuning can look like:

import tensorless as tl

print("Pretraining base model...")

base_model = tl.pretrain(
out="pretrained_base.tl",
max_seq_len=30
)

print("Base model pretrained -> pretrained_base.tl")

print("Fine-tuning on cat conversations...")

model = tl.train(
"data/conversations.json",
pretrained="pretrained_base.tl",
out="cat.tl",
)

print("CatTongue trained successfully!")

That's basically the experience I'm trying to create.

You don't need to write a massive training script.

You don't need thousands of lines just to get started.

You provide your data.

Tensorless handles the setup.

And if you want to customize things, you can.

Beginner vs Advanced User

This is something I really want Tensorless to get right.

If you're a beginner

You shouldn't have to understand every parameter.

You should be able to do something like:

model = tl.train("data/")

and get started.

If you're an advanced programmer

You should be able to configure the training yourself.

I want all the important configuration to be accessible in one place rather than making someone jump through a huge project just to change one setting.

So the idea is:

One simple function for beginners. Deep configuration for advanced users.

I Also Wanted It To Stay Lightweight

Another thing I'm trying to maintain is the overall package size.

Tensorless is currently under roughly 30–40 MB overall, rather than pulling in a huge collection of unnecessary components.

That's important to me because the original reason for this project was partly:

I want to experiment without needing a giant stack just to get started.

I don't want someone to install Tensorless for a small experiment and feel like they installed an entire operating system. 😭

Tensorless Is Now Actually Open Source

The project is available on GitHub and PyPI.

GitHub: https://github.com/DeveloperPuneet/Tensorless
PyPI: https://pypi.org/project/tensorless/

And CatTongue is also open source:

CatTongue: https://github.com/DeveloperPuneet/CatTongue

The CatTongue experiment is one of the things I'm using to test whether the training system is actually improving.

And Now I'm Stuck On The Hard Stuff

This is where the project is today.

The basic system works.

The API works.

Training works to a certain extent.

The automatic setup works.

But now I'm running into problems that aren't as easy as writing another Python function.

CUDA

I've been working on CUDA support and optimization.

The system can detect the GPU and show GPU-related activity, but I'm having problems getting the actual training workload to behave properly on the GPU.

Basically:

The GPU is detected, but it doesn't always mean the training is actually being accelerated the way it should be.

That's something I really want to fix.

TPU

I've also started working toward TPU support.

Again, detection is one thing.

Actually getting the training workload to execute correctly on the TPU is another.

So TPU support is currently something I want to improve substantially.

Multi-GPU

I've also been working on multi-GPU support.

The problem is that detecting multiple GPUs doesn't automatically mean the training workload is correctly distributed across them.

I want multi-GPU training to actually make use of the available devices rather than simply detecting them.

Training Results

This is probably the biggest issue.

The same training data can behave differently depending on the implementation.

My current training implementation doesn't always produce results comparable to established training frameworks.

The CatTongue experiment made that very obvious.

The model can learn words.

It can sometimes learn relationships between words.

But getting consistently good results is another level.

Automatic Configuration

The automatic configuration is also something I want to improve.

Right now, some decisions are based on predefined logic.

That's useful, but it isn't flexible enough for every dataset or model.

I want the system to become smarter about determining what configuration makes sense.

This Is Where I Need Help

And honestly, this is the main reason I'm writing this article.

I'm still learning.

I started this entire project because I was learning PyTorch.

I built two models with PyTorch.

Then I started automating the process.

Then I started building my own system.

And now I've reached areas where there are people who know way more than I do.

If you have experience with:

CUDA
GPU optimization
TPU/XLA
multi-GPU training
neural network implementation
training stability
data pipelines
model optimization
BPE/tokenization
machine-learning infrastructure

I'd genuinely love feedback.

You don't have to write code.

Even finding a problem and explaining why something is wrong would help.

And if you want to contribute code, even better.

The project is open source and the issues are there for exactly this reason.

What I Want To Build Next

I have a pretty ambitious list of things I'd like to experiment with.

Some of the ideas currently on my roadmap are:

Better CUDA optimization
Proper TPU training
Reliable multi-GPU training
Smarter automatic configuration
Improved data pipelines
More training/data formats
Additional model architectures
Game integration
CLI-based AI interaction
UI-based AI interaction
User-vs-AI experiences
Reinforcement learning
Q-learning
Deep Q-learning
And more

The goal isn't necessarily to implement everything immediately.

I want to keep expanding the system as I learn.

From Two PyTorch Models To Tensorless

Looking back, the progression is kind of funny.

I started with:

"I want to learn PyTorch."

Then I built two models with PyTorch.

Then:

"Why do I have to configure all this stuff every time?"

So I started automating it.

Then:

"Why is this getting so heavy?"

So I started trying to make it lightweight.

Then:

"What if I build more of the system myself?"

Then came:

CUDA.

TPUs.

Data pipelines.

BPE.

Multi-GPU.

And then eventually:

"Why does my cat suddenly speak ancient alien language?" 💀

That's basically how Tensorless happened.

I didn't start this project because I thought I could build something better than PyTorch.

I started it because I wanted to make one specific thing easier:

Take training data → configure as little as possible → train a model.

I wanted something where beginners could just give it their data and go, while advanced users could still control the system when they needed to.

And now I'm trying to figure out how far I can actually take that idea.

If You Want To Check It Out
Tensorless

https://github.com/DeveloperPuneet/Tensorless

https://pypi.org/project/tensorless/

CatTongue

https://github.com/DeveloperPuneet/CatTongue

If you're interested in the project, try it, break it, report something, suggest something, or contribute.

Especially if you understand the parts I'm currently struggling with.

I'm still learning.

That's kind of the whole reason this project exists.

I started by learning PyTorch.

Now I'm learning what it actually takes to build the stuff underneath it.

And honestly?

I'm curious to see where this goes. 🚀