• 0 Posts
  • 120 Comments
Joined 3 years ago
cake
Cake day: June 20th, 2023

help-circle






  • You actually have a lot of options in this space. You can take the prefill engine of just about any LLM and turn it’s transformer into a classifier by lobotomizing out the decoder. You can also use a diffusion model on single pass to surprisingly competent result.

    If Jeff looks easy enough to deploy by all means start there, but don’t discount the idea if Jeff sucks; you have a lot of options.


  • You could (and probably should) use a system-one style inference system for spam classification. Much cheaper and the structured output means it’s impossible to go rogue and curl some malware or whatever. It can absolutely misclassify but its output is programmatically structured and just ranks a pre-selected set of output tokens.

    In your case that’s

    Spam

    Not_spam



  • MIT: I build my thing on top of your thing, it’s my thing now. I sell it, you have no rights.

    GPL: I build my thing on top of your thing and your license forces me to distribute my thing under the GPL, including the source code.

    There are legitimate arguments for both. Sometimes it sucks when proprietary things get built on top of MIT code but sometimes it takes money for innovation, and in the realm of code the world often benefits anyway by virtue of copycats or distillation.


  • firefox does a lot of disk caching during regular use, mostly from webapps trying to be clever about using your disk space to cache chats, media, etc. there’s functionally nothing you can do about this per-site. It also does a lot of journaling in local sqlite databases for its own cookies, history, caching, etc.

    if you want actual solutions:

    use a DE like lxde to free up system memory as much as possible.

    move firefox’s caching and profile directories to /dev/shm and accept the memory footprint penalty on launch. accept that a crash or shutdown without flushing back to disk will result in losing this data.

    add an override to journald.conf.d to move the journal to volatile storage with a functionally nonexistant rolling size, but just big enough not to crash (~128mb)

    disable rsyslog

    create a systemd service to move /tmp /var/tmp /var/log and /var/cache/apt/archives to ramdisk on boot.

    disable any systemctl timer that you don’t absolutely need (apt-daily.timer apt-daily-upgrade.timer, logrotate.timer etc…)

    increase the journal commit interval on your root filesystem to something like ~60s


  • Well first, the linux kernel is open-source. This is not my opinion and everything I’m telling you is auditable.

    programs are generally written to wait for writes to close. There are good reasons for this. The simplest example I can think of would be a basic text editor’s “save and exit” function. It writes the data, and then closes. If it didn’t wait for the write to finish, and it closed mid-write, you’d lose half your data. Now remember that this doesn’t just apply to editors. It applies to logging functions, updating programs, creating a shortcut, changing a setting in a menu, etc… The program relies on kernel signaling to know when a file has “finished” writing. In the case of datacenter high-performance filesystems or raid controllers, it will use memory (think: ram) as cache for the writes and the kernel will report back to the process that the write is finished, even while a little bit of the file is still in ram and waiting to be flushed to disk. This is safe because those systems have multiple redundant power supplies and/or backup batteries. In the case of a desktop, you typically don’t do this because a power flicker or a system crash could take data with it. Modern SSDs make this somewhat of a non-issue because they’re still relatively fast at small writes while under load. The difference on a small write between a thrashing ssd and a thrashing hdd can be the difference between 200 nanoseconds (nvme ssd) and 15 milliseconds (hdd).

    Programs need to block on writes sometimes as a mater of procedure to ensure data integrity, and you’re hitting the worst case possible. 50,000-500,000 times slower than a modern enterprise filesystem, 1,000-10,000 slower than a bog-standard, modern consumer-grade nvme ssd.

    Operating systems are generally built with the assumption of mid-range consumer hardware from 4-6 years ago, and spinning disk root volumes haven’t been standard for over a decade. There are absolutely things you can do to make a system play nicely with a spinning root volume but modern general-purpose operating systems tune for different assumptions. Almost none of them tune for a 15ms write penalty on logging.



  • The Linux kernel (and windows btw) often lock threads to the completion of reads or writes to disk. When your primary drive is a spinner and you saturate it’s bandwidth with a long-running copy (your from windows copy) you take those normally short processes and add the latency penalty of a thrashing disk to every lock.

    Yes you can fix this by moving the entire kernel and operating system into ram at boot, but with 8gb you probably don’t want to do that, especially when a couple of Firefox tabs will fill that up pretty quickly.

    The solution is drop the gui, load the os into ram and disable logging, or be patient, and don’t expect any modern operating system to hold up when you are running long copies against a high-latency root disk.