• 0 Posts
  • 31 Comments
Joined 2 years ago
cake
Cake day: April 23rd, 2023

help-circle



  • Unfortunately I can’t even test Llama 3.1 in Alpaca because it refuses to download, showing some error message with the important bits cut off.

    That said, the Alpaca download interface seems much more robust, allowing me to select a model and then select any version of it for download, not just apparently picking whatever version it thinks I should use. That’s an improvement for sure. On GPT4All I basically have to download the model manually if I want one that’s not the default, and when I do that there’s a decent chance it doesn’t run on GPU.

    However, GPT4All allows me to plainly see how I can edit the system prompt and many other parameters the model is run with, and even configure multiple sets of parameters for the same model. That allows me to effectively pre-configure a model in much more creative ways, such as programming it to be a specific character with a specific background and mindset. I can get the Mistral model from earlier to act like anything from a very curt and emotionally neutral virtual intelligence named Jarvis to a grumpy fantasy monster whose behavior is transcribed by a narrator. GPT4All can even present an API endpoint to localhost for other programs to use.

    Alpaca seems to have some degree of model customization, but I can’t tell how well it compares, probably because I’m not familiar with using ollama and I don’t feel like tinkering with it since it doesn’t want to use my GPU. The one thing I can see that’s better in it is the use of multiple models at the same time; right now GPT4All will unload one model before it loads another.


  • I have a fairly substantial 16gb AMD GPU, and when I load in Llama 3.1 8B Instruct 128k (Q4_0), it gives me about 12 tokens per second. That’s reasonably fast enough for me, but only 50% faster than CPU (which I test by loading mlabonne’s abliterated Q4_K_M version, which runs on CPU in GPT4All, though I have no idea if that’s actually meant to be comparable in performance).

    Then I load in Nous Hermes 2 Mistral 7B DPO (also Q4_0) and it blazes through at 50+ tokens per second. So I don’t really know what’s going on there. Seems like performance varies a lot from model to model, but I don’t know enough to speculate why. I can’t even try Gemma2 models, GPT4All just crashes with them. I should probably test Alpaca to see if these perform any different there…




  • The ELI5 for Fedora’s atomic desktops is that if Windows had an Atomic Desktop version, Program Files and most of the Windows folder would be read only, and each program you installed yourself would go into its own folder in your user directory. That’s the basic idea. It’s harder to screw up an Atomic system as long as you stick to containerized app formats like flatpak/appimage whenever possible. It makes it easier for everyone to diagnose problems, and easier for users to roll back if an update has problems. Even if you were to install it right now, you could use one simple command to “roll back” to any image from the last three months.

    The benefit of Bazzite is you have all of the above, plus a lot of gaming-related stuff preinstalled which, if you were to install them yourself in a normal Fedora environment, you’d likely have to spend a lot of time just learning how they’re supposed to be configured, how they interact, which versions have problems, and how to troubleshoot problems when an update to one app breaks a prerequisite for something else; eventually you end up in config hell instead of actually using your computer. With Bazzite, the image maintainers are the ones in config hell - they work out the kinks, app versioning, communicate with upstream to fix issues, all that, so your system should be in the most functional state that a Linux system can be, so you only have to think about using your apps.

    tl;dr

    • Atomic Desktops are more resilient to randomly breaking from updates or user error, and are easier to revert to a prior state if problems do arise
    • Bazzite is a custom Atomic image with lots of gaming stuff preinstalled and preconfigured to work properly out of the box
    • If you’re a gamer and wanting to try out Linux, Bazzite is going to be the least painful way to get your feet wet.
    • Immutable distros are excellent for daily driving. I daily drive one myself!


  • My setup is not recommended, honestly. Old gaming PC from about 14 years ago with a couple extra hard drives, thrown in the closet with stripped-down Windows 10 on an old SSD, desktop version of Jellyfin, and an external drive for backups. Not even running in a Docker container because the CMOS battery is dead and getting to it is way too much of a hassle on that particular motherboard, so virtualization defaults to off whenever it completely loses power. Which it unfortunately does on occasion like winter storms, or summer heat, or if the wind is blowing.

    But hey, for the movies and shows we have on DVD/BD, as well as the music we’ve bought over the years, it does work for access from PCs and phones on the local network (Finamp + Jellyfin Media Player). I dabbled with IPTV for live TV replacement but found that only using totally free IPTV+metadata would take either much more work on no-virtualization Windows 10 than I’m willing to put up with, or have much more jank than my family is willing to put up with.







  • Onihikage@beehaw.orgtoLinux@lemmy.mlCrapped my system
    link
    fedilink
    English
    arrow-up
    2
    ·
    6 months ago

    Atomic means the core OS packages are in an immutable container such that none of its individual components can be updated separately; instead the entire container is replaced with a newer version when the system is updated. This makes it much less likely for something to break during normal use, and easier to rollback updates if something does happen to break. The ideal use case is a containerized environment where each app you use is installed in its own container, like Docker, or is otherwise self-contained such as flatpak installers, and doesn’t rely on any of the system’s packages.





  • When you’re keeping things in a tree structure for visual grouping and using containers to manage different logins, bookmarks will lose the tree structure, and you’ll have to specify which container to open it in. If your workflow involves a dozen tabs per context, locating the bookmarks and reopening them every time you switch contexts is a significant time and productivity loss.

    Consider the classic Evidence Board (also known as string wall, crazy wall, conspiracy board, etc.). Saving everything to bookmarks is the equivalent of putting your board’s contents into a drawer, then pinning everything back up whenever you need to look at or update that particular conspiracy. It works, but it’s cumbersome, error-prone, and wastes a lot of time; you’d only do this if you only have one board but multiple things to inspect. Leaving tabs open and simply unloading the inactive tab trees is like having multiple separate boards where you just roll them into a closet when you aren’t using them.