AI automation for newsletter

[ElhamAryanpur]

Hello!

After some conversation with Ozkriff and looking at how much work goes into editing the newsletter each month, I was wondering if it'd be a good idea to start using some automation tools.

For editorial roles, something like chatgpt/gpt4 can assist a lot. What I had in mind was the bot running on each pull request, and checking the content that was added and feed that to the AI for editing and auditing, and returning either a fixed version or list of things to do to fix them or update them.

For example one of my PR had too much repetition and extra information, and the title wasn't good. In such a case it can do all of that given the newsletter guidelines, or notify me to do them the way required.

Cost wise, since the newsletter is a monthly release, I don't think we can exceed a dollar at the busiest month given how cheap the api is, and since each section is small, the context of gpt3.5 won't be a problem either.

One other thing I remembered is that it can also assist in writing for projects that were announced but had no one to write for them, for example we had Rusty Jam #3. It can also write a section for it without taking time off the editors, and it can just be reviewed for fixes and added.

These are just my suggestions so I'm not sure if it can be appealing to use, I have some experience with OpenAI so I can assist in implementing it.

[erlend-sh]

I would love to see this experimented with.

The AI-assistance track is well worth exploring, but it's worth noting that this type of automation could also work with a much more basic feed aggregator that simply asks projects to list their update feeds (blog rss, mastodon, github releases etc.) and it'd create summaries for projects by simply linking out to their updates for the past month.

[ElhamAryanpur]

@erlend-sh that is actually a really really awesome idea!!!

[Vrixyz]

This week in rust has an interesting bot which might be worth investigating : https://github.com/extrawurst/twir-bot

[janhohenheim]

@ElhamAryanpur if you're still up for implementing something like this, I'd be very up for reviewing it and getting it merged :)
I also see some great time saving potential on the editing side and would gladly pay for the API access, since it would be really cheap.

The feature I'd like to see the most would be a short automated summary for content no one has written anything for yet. Maybe it's already enough to feed the raw HTML to GPT and ask it for a summary? I also know that there are services that do this kind of thing for you using GPT like https://notegpt.io/web-summary, idk if they're better than just entering our own prompt though.

[ElhamAryanpur]

@janhohenheim absolutely, since then I've invested a lot of time in my own side of LLM based software, and can say it's even better than ever to do something like this.

We can have four approaches:

1. using a custom model hosted on a VPS or similar. Provides full privacy, can be reused by other Rust based newsletter and publication, even social media moderation as well. periodically or automated, the locally hosted model fetch update changelogs and such, and does all the summaries and reports itself through RAG architecture.

2. using a custom or stock model hosted locally by a maintainer. Same as above, except This is very cost effective and pretty much free, no need for API access anywhere. Models like Mistral 0.2 7B has over 32k context length, 4.5GB in model file size (gguf), and can run on any modern computer. So pretty much anyone can use it. We can even use fine-tuned version such as hermes/dolphin mistral, for better results.

3. fine tune a model by a cloud provider such as OpenAI, claude, google, ... Bit expensive and at mercy of the cloud provider but it can have benefits of the first option.

4. using a stock model by a cloud provider such as OpenAI GPT 4, claude, gemini, ... Cheaper than third option but same risks. Bit of issue with these two options is degradation of the models over time as more gaurdrails are introduced and potentially can sometimes put a dent on your bank if suddenly price changes and such. They can also block your access on a whim if they like.

Personally I think second option would be the best to start with. RAG helps with auto search of changelogs and summary writing. Pull requests too but perhaps a bit difficult automatically locally than github actions 😅.

Let me know which options you'd think is nicer and I can begin.

[erlend-sh]

I also recommend checking out https://spiderwebai.xyz/ by @j-mendez

[janhohenheim]

@ElhamAryanpur great to hear! Since the newsletter has historically struggled with maintainer burden, I am more inclined to option 4. You know this stuff better than me though, so if you think that option 2 would be really really good for us, I'm ready to rent a cheap server on DigitalOcean and give you access.
Also, what do you think about the service @erlend-sh mentioned?

[ElhamAryanpur]

I also recommend checking out https://spiderwebai.xyz/ by @j-mendez

Yeah they're using RAG too, I assume langchain by most chances

[ElhamAryanpur]

@ElhamAryanpur great to hear! Since the newsletter has historically struggled with maintainer burden, I am more inclined to option 4. You know this stuff better than me though, so if you think that option 2 would be really really good for us, I'm ready to rent a cheap server on DigitalOcean and give you access.
Also, what do you think about the service @erlend-sh mentioned?

That is very true, I have wrote a section about my work there in the past and it shocked me how much work the maintainers did every month...

Yeah we can start locally for development, get some early testing on the newsletter, if the results were great, we can then move to hosting or keep it locally. I just don't wish to burden you for paying the servers or API 😅 trying to get a solution that anyone can use and contribute instead of hurting your wallet, especially at this stage

[janhohenheim]

@ElhamAryanpur alright then! Do you need anything from me to start? How do you want to organize yourself? If you create a repo with a readme on how to run the model, I can ensure it runs on my machine in the background (or on a machine I rented anyway to host a Minecraft server, hehe)

[ElhamAryanpur]

For sure, it'll probably be a repo. I'm not sure much on else yet, will keep you updated here. Thank you!

[j-mendez]

@ElhamAryanpur great to hear! Since the newsletter has historically struggled with maintainer burden, I am more inclined to option 4. You know this stuff better than me though, so if you think that option 2 would be really really good for us, I'm ready to rent a cheap server on DigitalOcean and give you access.
Also, what do you think about the service @erlend-sh mentioned?

That is very true, I have wrote a section about my work there in the past and it shocked me how much work the maintainers did every month...

Yeah we can start locally for development, get some early testing on the newsletter, if the results were great, we can then move to hosting or keep it locally. I just don't wish to burden you for paying the servers or API 😅 trying to get a solution that anyone can use and contribute instead of hurting your wallet, especially at this stage

Hi! If the bandwidth is minimal and simply a page or two ( it would take a lot of request to get to 1$ ), we also do not pad the cost for GPT from OpenAI. The dashboard is very early stage and being actively improved. The service is more flushed from an API perspective atm. I recommend testing a basic prompt on the GPT playground and works it works off a small set of HTML - use the GPT configuration to extract what is needed etc. Lmk if you have any questions. Thanks @erlend-sh!

[ElhamAryanpur]

Hm? We did talk about it in the options listed

[j-mendez]

Hm? We did talk about it in the options listed

If you create an account I can add a dollar to the account to experiment. The service goal is pretty much putting this project on a server to scale https://github.com/spider-rs/spider.