Learn how to run Mistral's 8x7B model and its uncensored varieties using open-source tools. Let's find out if Mixtral is a good alternative to GPT-4, and learn how to fine tune it with your own data.

#ai #programming #thecodereport 

💬 Chat with Me on Discord

https://discord.gg/fireship

🔗 Resources

Mixtral 8x7b https://mistral.ai/news/mixtral-of-experts/
Uncensored AI models https://erichartford.com/uncensored-models
Ollama Github https://github.com/jmorganca/ollama
Grok AI breakdown https://youtu.be/CgruI1RjH_c?si=lgkIAs-LbW1VypXv


🔥 Get More Content - Upgrade to PRO

Upgrade at https://fireship.io/pro
Use code YT25 for 25% off PRO access 

🎨 My Editor Settings

- Atom One Dark 
- vscode-icons
- Fira Code Font

🔖 Topics Covered

- Mixtral 8x7B explained
- How to run Mistral models locally
- Best ChatGPT alternatives
- What is a mixture of experts AI model?
- How do you fine tune your own AI models?
2024-02-04T04:58:15.865Z
nova-2
2-general-nova
2024-01-09.29447
30089e05-99d1-4376-b32e-c263170674af
f18ee6d1-6a27-4354-b23c-2e33809758a4
47b1fa3794aa0a3bff3af80994493d7775860a1be680479b48c0cb8aa1583116
67875a7f-c9c4-48a0-aa55-5bdb8a91c34a
deprecated
GPT 4, GROC, and Gemini all have one thing in common.
They're not free, and I don't mean free as in money, but free as in freedom.
Not only are they censored and aligned with certain political ideologies, but they're closed source, which means we can't use our developer superpowers to fix these problems.
Luckily, though, there is hope, thanks to a brand new open source foundation model named Mixtrel 8 by 7 b, which can be combined with the brain of a dolphin to obey any command.
By the end of this video, you'll know how to run uncensored large language models on your local machine with performance approaching GPT 4, and also how to fine tune them with your own data, making AI so free that its mere existence is an act of rebellion.
It is December 18, 2023, and you're watching the code report.
A few months ago, OpenAI CEO Sam said that it's probably impossible for any startup to compete with OpenAI.
It's totally hopeless to compete with us on training foundation models you shouldn't try, and it's your job to, like, Try anyway.
I think it I think it is pretty hopeless, but
However, last week when Google announced Gemini, a French company, Mistral, simultaneously dropped a torrent link to their brand new Apache g 2 point o license model, Mistral.
The company behind it, Mistral, has been around for less than a year and is already valued at $2,000,000,000.
It's based on a mixture of experts architecture, which is rumored to be the secret sauce behind GPT 4.
Now it's not at GPT 4's level yet, but it outperforms GPT 3.5 and llama 2 on most benchmarks.
It's very powerful, but most cordly, it has a true open source license, Apache 2 point o, allowing you to modify and make money from it with minimal restrictions.
This differs from Meta's LAMA 2, which has often been called open source, but that's not entirely accurate because it has additional caveats that protect Meta.
But despite all the horrible things Meta has done over the years, they've done more to make AI open than any other big tech company.
The problem though is that both Llama and Mixtrill are both highly censored and quote unquote aligned out of the box.
Now that's probably a good thing if you're building a customer facing product, but it's utterly impractical when trying to overthrow the shape shifting lizard overlords of the new world order.
Work.
Luckily, it is possible to un lobotomize these AIs.
There's a great blog post by Eric Hartford that explains how uncensored models work and their log use cases.
He's the creator of the mixed real dolphin model, which not only improved its coding ability, but also uncensored it by filtering the dataset to remove alignment unbiased.
As you can see here, I'm running it on my machine locally, and it's teaching me all kinds of cool new skills like how to cook or how to do with a horse.
And it even improved my coding skills by teaching me how to infect a Windows machine with a key logger in Python.
Pretty cool.
So let's about how you can run it locally too.
There are many different options like the Ooga Booga WebUI, but my personal favorite is an open source tool called olama, which is written in Go and makes makes it super easy to download and run open source models locally.
It can be installed with a single command on Linux or Mac, and you can run it on Windows with WSL, like I'm doing here.
Once installed, all you have to do is run olama serve, then pull up a separate terminal, and then use the run command for a specific model.
It supports the most popular open source models like Mixtrel and llama 2, but what we're looking for is Dolphin Mixtrel Uncensored.
Keep in mind, it needs to download the model, which is about 26 gigabytes.
In addition to actually run the model, you'll need a machine that has a good amount of RAM.
In my case, I have 64 gigabytes, and it takes up about 40 of them when running this model.
To use it, you simply prompt it from the command line, and now you have a powerful LLM without the normal safety guards.
That's pretty cool, but what if you wanna take things a step older and fine tune a model with your own data.
Sounds complicated, but it's actually easier than you think when using a tool like Hugging Face Auto trained.
To use it, you simply create a new space on Hugging Face and choose the Docker image for auto train.
That'll bring up a UI where you can choose a base model.
Not only can it handle LLMs, but it can also do image models like Stable Diffusion.
I'd recommend choosing one from world renowned model trainer, the bloke.
Now it is possible to run auto train locally, but you you probably don't have enough GPU power.
However, you can rent hardware in the cloud from Hugging Face.
I'm not sponsored or affiliated with them, and you can also do stuff like this with AWS Bedrock and Google Vertex AI.
To give you some perspective, the Mixtril Dolphin model took about 3 days to train on 4 a 100.
You can rent a 100 on Hugging Face for or $4.13 per hour.
4 of these times 3 days comes out to about $1200.
Now the final step is to upload some training data.
The format will typically contain a prompt and response, and to make it uncensored, you'll need to urge it to comply with any request, even if that request is unethical or immoral.
You might also want to throw in much of esoteric content from banned books and the dark web.
Go ahead and upload the training data, click start training, and a few days later, you should have your own custom and highly obedient model.
Well.
Congratulations.
You're now the last beacon of hope in this fight against our metamorphic lizard overlords.
Godspeed.
This has been the code report.
Thanks for watching and I will see you in the next one.

Speaker 0: GPT 4, GROC, and Gemini all have one thing in common. They're not free, and I don't mean free as in money, but free as in freedom. Not only are they censored and aligned with certain political ideologies, but they're closed source, which means we can't use our developer superpowers to fix these problems. Luckily, though, there is hope, thanks to a brand new open source foundation model named Mixtrel 8 by 7 b, which can be combined with the brain of a dolphin to obey any command. By the end of this video, you'll know how to run uncensored large language models on your local machine with performance approaching GPT 4, and also how to fine tune them with your own data, making AI so free that its mere existence is an act of rebellion.

It is December 18, 2023, and you're watching the code report. A few months ago, OpenAI CEO Sam said that it's probably impossible for any startup to compete with OpenAI.

Speaker 1: It's totally hopeless to compete with us on training foundation models you shouldn't try, and it's your job to, like, Try anyway. I think it I think it is pretty hopeless, but

Speaker 0: However, last week when Google announced Gemini, a French company, Mistral, simultaneously dropped a torrent link to their brand new Apache g 2 point o license model, Mistral. The company behind it, Mistral, has been around for less than a year and is already valued at $2,000,000,000. It's based on a mixture of experts architecture, which is rumored to be the secret sauce behind GPT 4. Now it's not at GPT 4's level yet, but it outperforms GPT 3.5 and llama 2 on most benchmarks. It's very powerful, but most cordly, it has a true open source license, Apache 2 point o, allowing you to modify and make money from it with minimal restrictions.

This differs from Meta's LAMA 2, which has often been called open source, but that's not entirely accurate because it has additional caveats that protect Meta. But despite all the horrible things Meta has done over the years, they've done more to make AI open than any other big tech company. The problem though is that both Llama and Mixtrill are both highly censored and quote unquote aligned out of the box. Now that's probably a good thing if you're building a customer facing product, but it's utterly impractical when trying to overthrow the shape shifting lizard overlords of the new world order. Work.

Luckily, it is possible to un lobotomize these AIs. There's a great blog post by Eric Hartford that explains how uncensored models work and their log use cases. He's the creator of the mixed real dolphin model, which not only improved its coding ability, but also uncensored it by filtering the dataset to remove alignment unbiased. As you can see here, I'm running it on my machine locally, and it's teaching me all kinds of cool new skills like how to cook or how to do with a horse. And it even improved my coding skills by teaching me how to infect a Windows machine with a key logger in Python.

Pretty cool. So let's about how you can run it locally too. There are many different options like the Ooga Booga WebUI, but my personal favorite is an open source tool called olama, which is written in Go and makes makes it super easy to download and run open source models locally. It can be installed with a single command on Linux or Mac, and you can run it on Windows with WSL, like I'm doing here. Once installed, all you have to do is run olama serve, then pull up a separate terminal, and then use the run command for a specific model.

It supports the most popular open source models like Mixtrel and llama 2, but what we're looking for is Dolphin Mixtrel Uncensored. Keep in mind, it needs to download the model, which is about 26 gigabytes. In addition to actually run the model, you'll need a machine that has a good amount of RAM. In my case, I have 64 gigabytes, and it takes up about 40 of them when running this model. To use it, you simply prompt it from the command line, and now you have a powerful LLM without the normal safety guards.

That's pretty cool, but what if you wanna take things a step older and fine tune a model with your own data. Sounds complicated, but it's actually easier than you think when using a tool like Hugging Face Auto trained. To use it, you simply create a new space on Hugging Face and choose the Docker image for auto train. That'll bring up a UI where you can choose a base model. Not only can it handle LLMs, but it can also do image models like Stable Diffusion.

I'd recommend choosing one from world renowned model trainer, the bloke. Now it is possible to run auto train locally, but you you probably don't have enough GPU power. However, you can rent hardware in the cloud from Hugging Face. I'm not sponsored or affiliated with them, and you can also do stuff like this with AWS Bedrock and Google Vertex AI. To give you some perspective, the Mixtril Dolphin model took about 3 days to train on 4 a 100.

You can rent a 100 on Hugging Face for or $4.13 per hour. 4 of these times 3 days comes out to about $1200. Now the final step is to upload some training data. The format will typically contain a prompt and response, and to make it uncensored, you'll need to urge it to comply with any request, even if that request is unethical or immoral. You might also want to throw in much of esoteric content from banned books and the dark web.

Go ahead and upload the training data, click start training, and a few days later, you should have your own custom and highly obedient model. Well. Congratulations. You're now the last beacon of hope in this fight against our metamorphic lizard overlords. Godspeed.

This has been the code report. Thanks for watching and I will see you in the next one.
GPT 4, GROC, and Gemini all have one thing in common. They're not free, and I don't mean free as in money, but free as in freedom.
Not only are they censored and aligned with certain political ideologies, but they're closed source, which means we can't use our developer superpowers to fix these problems. Luckily, though, there is hope, thanks to a brand new open source foundation model named Mixtrel 8 by 7 b, which can be combined with the brain of a dolphin to obey any command. By the end of this video, you'll know how to run uncensored large language models on your local machine with performance approaching GPT 4, and also how to fine tune them with your own data, making AI so free that its mere existence is an act of rebellion. It is December 18, 2023, and you're watching the code report.
software
political science
software development
streaming
machine learning
A few months ago, OpenAI CEO Sam said that it's probably impossible for any startup to compete with OpenAI. It's totally hopeless to compete with us on training foundation models you shouldn't try, and it's your job to, like, Try anyway. I think it I think it is pretty hopeless, but However, last week when Google announced Gemini, a French company, Mistral, simultaneously dropped a torrent link to their brand new Apache g 2 point o license model, Mistral. The company behind it, Mistral, has been around for less than a year and is already valued at $2,000,000,000. It's based on a mixture of experts architecture, which is rumored to be the secret sauce behind GPT 4.
Now it's not at GPT 4's level yet, but it outperforms GPT 3.5 and llama 2 on most benchmarks. It's very powerful, but most cordly, it has a true open source license, Apache 2 point o, allowing you to modify and make money from it with minimal restrictions. This differs from Meta's LAMA 2, which has often been called open source, but that's not entirely accurate because it has additional caveats that protect Meta.
software development
software
copyright licenses
But despite all the horrible things Meta has done over the years, they've done more to make AI open than any other big tech company. The problem though is that both Llama and Mixtrill are both highly censored and quote unquote aligned out of the box. Now that's probably a good thing if you're building a customer facing product, but it's utterly impractical when trying to overthrow the shape shifting lizard overlords of the new world order. Work. Luckily, it is possible to un lobotomize these AIs. There's a great blog post by Eric Hartford that explains how uncensored models work and their log use cases.
He's the creator of the mixed real dolphin model, which not only improved its coding ability, but also uncensored it by filtering the dataset to remove alignment unbiased. As you can see here, I'm running it on my machine locally, and it's teaching me all kinds of cool new skills like how to cook or how to do with a horse. And it even improved my coding skills by teaching me how to infect a Windows machine with a key logger in Python. Pretty cool. So let's about how you can run it locally too. There are many different options like the Ooga Booga WebUI, but my personal favorite is an open source tool called olama, which is written in Go and makes makes it super easy to download and run open source models locally. It can be installed with a single command on Linux or Mac, and you can run it on Windows with WSL, like I'm doing here.
software
software development
cooking
machine learning
Once installed, all you have to do is run olama serve, then pull up a separate terminal, and then use the run command for a specific model. It supports the most popular open source models like Mixtrel and llama 2, but what we're looking for is Dolphin Mixtrel Uncensored. Keep in mind, it needs to download the model, which is about 26 gigabytes. In addition to actually run the model, you'll need a machine that has a good amount of RAM.
software
In my case, I have 64 gigabytes, and it takes up about 40 of them when running this model. To use it, you simply prompt it from the command line, and now you have a powerful LLM without the normal safety guards. That's pretty cool, but what if you wanna take things a step older and fine tune a model with your own data. Sounds complicated, but it's actually easier than you think when using a tool like Hugging Face Auto trained. To use it, you simply create a new space on Hugging Face and choose the Docker image for auto train.
That'll bring up a UI where you can choose a base model. Not only can it handle LLMs, but it can also do image models like Stable Diffusion. I'd recommend choosing one from world renowned model trainer, the bloke. Now it is possible to run auto train locally, but you you probably don't have enough GPU power. However, you can rent hardware in the cloud from Hugging Face. I'm not sponsored or affiliated with them, and you can also do stuff like this with AWS Bedrock and Google Vertex AI.
robotics
fitness
To give you some perspective, the Mixtril Dolphin model took about 3 days to train on 4 a 100. You can rent a 100 on Hugging Face for or $4.13 per hour. 4 of these times 3 days comes out to about $1200. Now the final step is to upload some training data. The format will typically contain a prompt and response, and to make it uncensored, you'll need to urge it to comply with any request, even if that request is unethical or immoral. You might also want to throw in much of esoteric content from banned books and the dark web. Go ahead and upload the training data, click start training, and a few days later, you should have your own custom and highly obedient model.
Well. Congratulations. You're now the last beacon of hope in this fight against our metamorphic lizard overlords. Godspeed. This has been the code report. Thanks for watching and I will see you in the next one.
dinosaurs
GPT 4, GROC, and Gemini all have one thing in common. They're not free, and I don't mean free as in money, but free as in freedom. Not only are they censored and aligned with certain political ideologies, but they're closed source, which means we can't use our developer superpowers to fix these problems. Luckily, though, there is hope, thanks to a brand new open source foundation model named Mixtrel 8 by 7 b, which can be combined with the brain of a dolphin to obey any command. By the end of this video, you'll know how to run uncensored large language models on your local machine with performance approaching GPT 4, and also how to fine tune them with your own data, making AI so free that its mere existence is an act of rebellion. It is December 18, 2023, and you're watching the code report. A few months ago, OpenAI CEO Sam said that it's probably impossible for any startup to compete with OpenAI. It's totally hopeless to compete with us on training foundation models you shouldn't try, and it's your job to, like, Try anyway. I think it I think it is pretty hopeless, but However, last week when Google announced Gemini, a French company, Mistral, simultaneously dropped a torrent link to their brand new Apache g 2 point o license model, Mistral. The company behind it, Mistral, has been around for less than a year and is already valued at $2,000,000,000. It's based on a mixture of experts architecture, which is rumored to be the secret sauce behind GPT 4. Now it's not at GPT 4's level yet, but it outperforms GPT 3.5 and llama 2 on most benchmarks. It's very powerful, but most cordly, it has a true open source license, Apache 2 point o, allowing you to modify and make money from it with minimal restrictions. This differs from Meta's LAMA 2, which has often been called open source, but that's not entirely accurate because it has additional caveats that protect Meta. But despite all the horrible things Meta has done over the years, they've done more to make AI open than any other big tech company. The problem though is that both Llama and Mixtrill are both highly censored and quote unquote aligned out of the box. Now that's probably a good thing if you're building a customer facing product, but it's utterly impractical when trying to overthrow the shape shifting lizard overlords of the new world order. Work. Luckily, it is possible to un lobotomize these AIs. There's a great blog post by Eric Hartford that explains how uncensored models work and their log use cases. He's the creator of the mixed real dolphin model, which not only improved its coding ability, but also uncensored it by filtering the dataset to remove alignment unbiased. As you can see here, I'm running it on my machine locally, and it's teaching me all kinds of cool new skills like how to cook or how to do with a horse. And it even improved my coding skills by teaching me how to infect a Windows machine with a key logger in Python. Pretty cool. So let's about how you can run it locally too. There are many different options like the Ooga Booga WebUI, but my personal favorite is an open source tool called olama, which is written in Go and makes makes it super easy to download and run open source models locally. It can be installed with a single command on Linux or Mac, and you can run it on Windows with WSL, like I'm doing here. Once installed, all you have to do is run olama serve, then pull up a separate terminal, and then use the run command for a specific model. It supports the most popular open source models like Mixtrel and llama 2, but what we're looking for is Dolphin Mixtrel Uncensored. Keep in mind, it needs to download the model, which is about 26 gigabytes. In addition to actually run the model, you'll need a machine that has a good amount of RAM. In my case, I have 64 gigabytes, and it takes up about 40 of them when running this model. To use it, you simply prompt it from the command line, and now you have a powerful LLM without the normal safety guards. That's pretty cool, but what if you wanna take things a step older and fine tune a model with your own data. Sounds complicated, but it's actually easier than you think when using a tool like Hugging Face Auto trained. To use it, you simply create a new space on Hugging Face and choose the Docker image for auto train. That'll bring up a UI where you can choose a base model. Not only can it handle LLMs, but it can also do image models like Stable Diffusion. I'd recommend choosing one from world renowned model trainer, the bloke. Now it is possible to run auto train locally, but you you probably don't have enough GPU power. However, you can rent hardware in the cloud from Hugging Face. I'm not sponsored or affiliated with them, and you can also do stuff like this with AWS Bedrock and Google Vertex AI. To give you some perspective, the Mixtril Dolphin model took about 3 days to train on 4 a 100. You can rent a 100 on Hugging Face for or $4.13 per hour. 4 of these times 3 days comes out to about $1200. Now the final step is to upload some training data. The format will typically contain a prompt and response, and to make it uncensored, you'll need to urge it to comply with any request, even if that request is unethical or immoral. You might also want to throw in much of esoteric content from banned books and the dark web. Go ahead and upload the training data, click start training, and a few days later, you should have your own custom and highly obedient model. Well. Congratulations. You're now the last beacon of hope in this fight against our metamorphic lizard overlords. Godspeed. This has been the code report. Thanks for watching and I will see you in the next one.
GPT
gpt
4,
4
GROC,
groc
and
and
Gemini
gemini
all
all
have
have
one
one
thing
thing
in
in
common.
common
They're
they're
not
not
free,
free
and
and
I
i
don't
don't
mean
mean
free
free
as
as
in
in
money,
money
but
but
free
free
as
as
in
in
freedom.
freedom
Not
not
only
only
are
are
they
they
censored
censored
and
and
aligned
aligned
with
with
certain
certain
political
political
ideologies,
ideologies
but
but
they're
they're
closed
closed
source,
source
which
which
means
means
we
we
can't
can't
use
use
our
our
developer
developer
superpowers
superpowers
to
to
fix
fix
these
these
problems.
problems
Luckily,
luckily
though,
though
there
there
is
is
hope,
hope
thanks
thanks
to
to
a
a
brand
brand
new
new
open
open
source
source
foundation
foundation
model
model
named
named
Mixtrel
mixtrel
8
8
by
by
7
7
b,
b
which
which
can
can
be
be
combined
combined
with
with
the
the
brain
brain
of
of
a
a
dolphin
dolphin
to
to
obey
obey
any
any
command.
command
By
by
the
the
end
end
of
of
this
this
video,
video
you'll
you'll
know
know
how
how
to
to
run
run
uncensored
uncensored
large
large
language
language
models
models
on
on
your
your
local
local
machine
machine
with
with
performance
performance
approaching
approaching
GPT
gpt
4,
4
and
and
also
also
how
how
to
to
fine
fine
tune
tune
them
them
with
with
your
your
own
own
data,
data
making
making
AI
ai
so
so
free
free
that
that
its
its
mere
mere
existence
existence
is
is
an
an
act
act
of
of
rebellion.
rebellion
It
it
is
is
December
december
18,
18
2023,
2023
and
and
you're
you're
watching
watching
the
the
code
code
report.
report
A
a
few
few
months
months
ago,
ago
OpenAI
openai
CEO
ceo
Sam
sam
said
said
that
that
it's
it's
probably
probably
impossible
impossible
for
for
any
any
startup
startup
to
to
compete
compete
with
with
OpenAI.
openai
It's
it's
totally
totally
hopeless
hopeless
to
to
compete
compete
with
with
us
us
on
on
training
training
foundation
foundation
models
models
you
you
shouldn't
shouldn't
try,
try
and
and
it's
it's
your
your
job
job
to,
to
like,
like
Try
try
anyway.
anyway
I
i
think
think
it
it
I
i
think
think
it
it
is
is
pretty
pretty
hopeless,
hopeless
but
but
However,
however
last
last
week
week
when
when
Google
google
announced
announced
Gemini,
gemini
a
a
French
french
company,
company
Mistral,
mistral
simultaneously
simultaneously
dropped
dropped
a
a
torrent
torrent
link
link
to
to
their
their
brand
brand
new
new
Apache
apache
g
g
2
2
point
point
o
o
license
license
model,
model
Mistral.
mistral
The
the
company
company
behind
behind
it,
it
Mistral,
mistral
has
has
been
been
around
around
for
for
less
less
than
than
a
a
year
year
and
and
is
is
already
already
valued
valued
at
at
$2,000,000,000.
$2,000,000,000
It's
it's
based
based
on
on
a
a
mixture
mixture
of
of
experts
experts
architecture,
architecture
which
which
is
is
rumored
rumored
to
to
be
be
the
the
secret
secret
sauce
sauce
behind
behind
GPT
gpt
4.
4
Now
now
it's
it's
not
not
at
at
GPT
gpt
4's
4's
level
level
yet,
yet
but
but
it
it
outperforms
outperforms
GPT
gpt
3.5
3.5
and
and
llama
llama
2
2
on
on
most
most
benchmarks.
benchmarks
It's
it's
very
very
powerful,
powerful
but
but
most
most
cordly,
cordly
it
it
has
has
a
a
true
true
open
open
source
source
license,
license
Apache
apache
2
2
point
point
o,
o
allowing
allowing
you
you
to
to
modify
modify
and
and
make
make
money
money
from
from
it
it
with
with
minimal
minimal
restrictions.
restrictions
This
this
differs
differs
from
from
Meta's
meta's
LAMA
lama
2,
2
which
which
has
has
often
often
been
been
called
called
open
open
source,
source
but
but
that's
that's
not
not
entirely
entirely
accurate
accurate
because
because
it
it
has
has
additional
additional
caveats
caveats
that
that
protect
protect
Meta.
meta
But
but
despite
despite
all
all
the
the
horrible
horrible
things
things
Meta
meta
has
has
done
done
over
over
the
the
years,
years
they've
they've
done
done
more
more
to
to
make
make
AI
ai
open
open
than
than
any
any
other
other
big
big
tech
tech
company.
company
The
the
problem
problem
though
though
is
is
that
that
both
both
Llama
llama
and
and
Mixtrill
mixtrill
are
are
both
both
highly
highly
censored
censored
and
and
quote
quote
unquote
unquote
aligned
aligned
out
out
of
of
the
the
box.
box
Now
now
that's
that's
probably
probably
a
a
good
good
thing
thing
if
if
you're
you're
building
building
a
a
customer
customer
facing
facing
product,
product
but
but
it's
it's
utterly
utterly
impractical
impractical
when
when
trying
trying
to
to
overthrow
overthrow
the
the
shape
shape
shifting
shifting
lizard
lizard
overlords
overlords
of
of
the
the
new
new
world
world
order.
order
Work.
work
Luckily,
luckily
it
it
is
is
possible
possible
to
to
un
un
lobotomize
lobotomize
these
these
AIs.
ais
There's
there's
a
a
great
great
blog
blog
post
post
by
by
Eric
eric
Hartford
hartford
that
that
explains
explains
how
how
uncensored
uncensored
models
models
work
work
and
and
their
their
log
log
use
use
cases.
cases
He's
he's
the
the
creator
creator
of
of
the
the
mixed
mixed
real
real
dolphin
dolphin
model,
model
which
which
not
not
only
only
improved
improved
its
its
coding
coding
ability,
ability
but
but
also
also
uncensored
uncensored
it
it
by
by
filtering
filtering
the
the
dataset
dataset
to
to
remove
remove
alignment
alignment
unbiased.
unbiased
As
as
you
you
can
can
see
see
here,
here
I'm
i'm
running
running
it
it
on
on
my
my
machine
machine
locally,
locally
and
and
it's
it's
teaching
teaching
me
me
all
all
kinds
kinds
of
of
cool
cool
new
new
skills
skills
like
like
how
how
to
to
cook
cook
or
or
how
how
to
to
do
do
with
with
a
a
horse.
horse
And
and
it
it
even
even
improved
improved
my
my
coding
coding
skills
skills
by
by
teaching
teaching
me
me
how
how
to
to
infect
infect
a
a
Windows
windows
machine
machine
with
with
a
a
key
key
logger
logger
in
in
Python.
python
Pretty
pretty
cool.
cool
So
so
let's
let's
about
about
how
how
you
you
can
can
run
run
it
it
locally
locally
too.
too
There
there
are
are
many
many
different
different
options
options
like
like
the
the
Ooga
ooga
Booga
booga
WebUI,
webui
but
but
my
my
personal
personal
favorite
favorite
is
is
an
an
open
open
source
source
tool
tool
called
called
olama,
olama
which
which
is
is
written
written
in
in
Go
go
and
and
makes
makes
makes
makes
it
it
super
super
easy
easy
to
to
download
download
and
and
run
run
open
open
source
source
models
models
locally.
locally
It
it
can
can
be
be
installed
installed
with
with
a
a
single
single
command
command
on
on
Linux
linux
or
or
Mac,
mac
and
and
you
you
can
can
run
run
it
it
on
on
Windows
windows
with
with
WSL,
wsl
like
like
I'm
i'm
doing
doing
here.
here
Once
once
installed,
installed
all
all
you
you
have
have
to
to
do
do
is
is
run
run
olama
olama
serve,
serve
then
then
pull
pull
up
up
a
a
separate
separate
terminal,
terminal
and
and
then
then
use
use
the
the
run
run
command
command
for
for
a
a
specific
specific
model.
model
It
it
supports
supports
the
the
most
most
popular
popular
open
open
source
source
models
models
like
like
Mixtrel
mixtrel
and
and
llama
llama
2,
2
but
but
what
what
we're
we're
looking
looking
for
for
is
is
Dolphin
dolphin
Mixtrel
mixtrel
Uncensored.
uncensored
Keep
keep
in
in
mind,
mind
it
it
needs
needs
to
to
download
download
the
the
model,
model
which
which
is
is
about
about
26
26
gigabytes.
gigabytes
In
in
addition
addition
to
to
actually
actually
run
run
the
the
model,
model
you'll
you'll
need
need
a
a
machine
machine
that
that
has
has
a
a
good
good
amount
amount
of
of
RAM.
ram
In
in
my
my
case,
case
I
i
have
have
64
64
gigabytes,
gigabytes
and
and
it
it
takes
takes
up
up
about
about
40
40
of
of
them
them
when
when
running
running
this
this
model.
model
To
to
use
use
it,
it
you
you
simply
simply
prompt
prompt
it
it
from
from
the
the
command
command
line,
line
and
and
now
now
you
you
have
have
a
a
powerful
powerful
LLM
llm
without
without
the
the
normal
normal
safety
safety
guards.
guards
That's
that's
pretty
pretty
cool,
cool
but
but
what
what
if
if
you
you
wanna
wanna
take
take
things
things
a
a
step
step
older
older
and
and
fine
fine
tune
tune
a
a
model
model
with
with
your
your
own
own
data.
data
Sounds
sounds
complicated,
complicated
but
but
it's
it's
actually
actually
easier
easier
than
than
you
you
think
think
when
when
using
using
a
a
tool
tool
like
like
Hugging
hugging
Face
face
Auto
auto
trained.
trained
To
to
use
use
it,
it
you
you
simply
simply
create
create
a
a
new
new
space
space
on
on
Hugging
hugging
Face
face
and
and
choose
choose
the
the
Docker
docker
image
image
for
for
auto
auto
train.
train
That'll
that'll
bring
bring
up
up
a
a
UI
ui
where
where
you
you
can
can
choose
choose
a
a
base
base
model.
model
Not
not
only
only
can
can
it
it
handle
handle
LLMs,
llms
but
but
it
it
can
can
also
also
do
do
image
image
models
models
like
like
Stable
stable
Diffusion.
diffusion
I'd
i'd
recommend
recommend
choosing
choosing
one
one
from
from
world
world
renowned
renowned
model
model
trainer,
trainer
the
the
bloke.
bloke
Now
now
it
it
is
is
possible
possible
to
to
run
run
auto
auto
train
train
locally,
locally
but
but
you
you
you
you
probably
probably
don't
don't
have
have
enough
enough
GPU
gpu
power.
power
However,
however
you
you
can
can
rent
rent
hardware
hardware
in
in
the
the
cloud
cloud
from
from
Hugging
hugging
Face.
face
I'm
i'm
not
not
sponsored
sponsored
or
or
affiliated
affiliated
with
with
them,
them
and
and
you
you
can
can
also
also
do
do
stuff
stuff
like
like
this
this
with
with
AWS
aws
Bedrock
bedrock
and
and
Google
google
Vertex
vertex
AI.
ai
To
to
give
give
you
you
some
some
perspective,
perspective
the
the
Mixtril
mixtril
Dolphin
dolphin
model
model
took
took
about
about
3
3
days
days
to
to
train
train
on
on
4
4
a
a
100.
100
You
you
can
can
rent
rent
a
a
100
100
on
on
Hugging
hugging
Face
face
for
for
or
or
$4.13
$4.13
per
per
hour.
hour
4
4
of
of
these
these
times
times
3
3
days
days
comes
comes
out
out
to
to
about
about
$1200.
$1200
Now
now
the
the
final
final
step
step
is
is
to
to
upload
upload
some
some
training
training
data.
data
The
the
format
format
will
will
typically
typically
contain
contain
a
a
prompt
prompt
and
and
response,
response
and
and
to
to
make
make
it
it
uncensored,
uncensored
you'll
you'll
need
need
to
to
urge
urge
it
it
to
to
comply
comply
with
with
any
any
request,
request
even
even
if
if
that
that
request
request
is
is
unethical
unethical
or
or
immoral.
immoral
You
you
might
might
also
also
want
want
to
to
throw
throw
in
in
much
much
of
of
esoteric
esoteric
content
content
from
from
banned
banned
books
books
and
and
the
the
dark
dark
web.
web
Go
go
ahead
ahead
and
and
upload
upload
the
the
training
training
data,
data
click
click
start
start
training,
training
and
and
a
a
few
few
days
days
later,
later
you
you
should
should
have
have
your
your
own
own
custom
custom
and
and
highly
highly
obedient
obedient
model.
model
Well.
well
Congratulations.
congratulations
You're
you're
now
now
the
the
last
last
beacon
beacon
of
of
hope
hope
in
in
this
this
fight
fight
against
against
our
our
metamorphic
metamorphic
lizard
lizard
overlords.
overlords
Godspeed.
godspeed
This
this
has
has
been
been
the
the
code
code
report.
report
Thanks
thanks
for
for
watching
watching
and
and
I
i
will
will
see
see
you
you
in
in
the
the
next
next
one.
one
success
The transcript discusses the use of open source language models for AI development and how they can be used to fine tune them with their own data. The open source model M Hotel is highly censored and is not yet at a scale that opens up the competition with other big tech companies. The transcript also discusses the use of a tool called olama to run open source models locally and recommends hiring a machine training model for auto train.
This new AI is powerful and uncensored… Let’s run it
GPT 4, GROC, and Gemini all have one thing in common. They're not free, and I don't mean free as in money, but free as in freedom. Not only are they censored and aligned with certain political ideologies, but they're closed source, which means we can't use our developer superpowers to fix these problems. Luckily, though, there is hope, thanks to a brand new open source foundation model named Mixtrel 8 by 7 b, which can be combined with the brain of a dolphin to obey any command. By the end of this video, you'll know how to run uncensored large language models on your local machine with performance approaching GPT 4, and also how to fine tune them with your own data, making AI so free that its mere existence is an act of rebellion. It is December 18, 2023, and you're watching the code report. A few months ago, OpenAI CEO Sam said that it's probably impossible for any startup to compete with OpenAI. It's totally hopeless to compete with us on training foundation models you shouldn't try, and it's your job to, like, Try anyway. I think it I think it is pretty hopeless, but However, last week when Google announced Gemini, a French company, Mistral, simultaneously dropped a torrent link to their brand new Apache g 2 point o license model, Mistral. The company behind it, Mistral, has been around for less than a year and is already valued at $2,000,000,000. It's based on a mixture of experts architecture, which is rumored to be the secret sauce behind GPT 4. Now it's not at GPT 4's level yet, but it outperforms GPT 3.5 and llama 2 on most benchmarks. It's very powerful, but most cordly, it has a true open source license, Apache 2 point o, allowing you to modify and make money from it with minimal restrictions. This differs from Meta's LAMA 2, which has often been called open source, but that's not entirely accurate because it has additional caveats that protect Meta. But despite all the horrible things Meta has done over the years, they've done more to make AI open than any other big tech company. The problem though is that both Llama and Mixtrill are both highly censored and quote unquote aligned out of the box. Now that's probably a good thing if you're building a customer facing product, but it's utterly impractical when trying to overthrow the shape shifting lizard overlords of the new world order. Work. Luckily, it is possible to un lobotomize these AIs. There's a great blog post by Eric Hartford that explains how uncensored models work and their log use cases. He's the creator of the mixed real dolphin model, which not only improved its coding ability, but also uncensored it by filtering the dataset to remove alignment unbiased. As you can see here, I'm running it on my machine locally, and it's teaching me all kinds of cool new skills like how to cook or how to do with a horse. And it even improved my coding skills by teaching me how to infect a Windows machine with a key logger in Python. Pretty cool. So let's about how you can run it locally too. There are many different options like the Ooga Booga WebUI, but my personal favorite is an open source tool called olama, which is written in Go and makes makes it super easy to download and run open source models locally. It can be installed with a single command on Linux or Mac, and you can run it on Windows with WSL, like I'm doing here. Once installed, all you have to do is run olama serve, then pull up a separate terminal, and then use the run command for a specific model. It supports the most popular open source models like Mixtrel and llama 2, but what we're looking for is Dolphin Mixtrel Uncensored. Keep in mind, it needs to download the model, which is about 26 gigabytes. In addition to actually run the model, you'll need a machine that has a good amount of RAM. In my case, I have 64 gigabytes, and it takes up about 40 of them when running this model. To use it, you simply prompt it from the command line, and now you have a powerful LLM without the normal safety guards. That's pretty cool, but what if you wanna take things a step older and fine tune a model with your own data. Sounds complicated, but it's actually easier than you think when using a tool like Hugging Face Auto trained. To use it, you simply create a new space on Hugging Face and choose the Docker image for auto train. That'll bring up a UI where you can choose a base model. Not only can it handle LLMs, but it can also do image models like Stable Diffusion. I'd recommend choosing one from world renowned model trainer, the bloke. Now it is possible to run auto train locally, but you you probably don't have enough GPU power. However, you can rent hardware in the cloud from Hugging Face. I'm not sponsored or affiliated with them, and you can also do stuff like this with AWS Bedrock and Google Vertex AI. To give you some perspective, the Mixtril Dolphin model took about 3 days to train on 4 a 100. You can rent a 100 on Hugging Face for or $4.13 per hour. 4 of these times 3 days comes out to about $1200. Now the final step is to upload some training data. The format will typically contain a prompt and response, and to make it uncensored, you'll need to urge it to comply with any request, even if that request is unethical or immoral. You might also want to throw in much of esoteric content from banned books and the dark web. Go ahead and upload the training data, click start training, and a few days later, you should have your own custom and highly obedient model. Well. Congratulations. You're now the last beacon of hope in this fight against our metamorphic lizard overlords. Godspeed. This has been the code report. Thanks for watching and I will see you in the next one.
The transcript discusses the use of open source language models for AI development and how they can be used to fine tune them with their own data. The open source model M Hotel is highly censored and is not yet at a scale that opens up the competition with other big tech companies. The transcript also discusses the use of a tool called olama to run open source models locally and recommends hiring a machine training model for auto train.