Load Tests Python Task Queues
What My Project Does
While looking for task queues, I found that there are many options available in the Python ecosystem, making it really hard to choose the right one. To get a sense of how each library performs and to help make an informed decision, I conducted a load test on some of the most popular ones: Python-RQ, ARQ, Celery, Huey, and Dramatiq.
Target Audience
I hope my findings can help those who are also looking for a task queue solution in Python.
Comparison
Most articles out there seem to focus on comparing the features of these libraries but rarely discuss performance. While there could be a lot of improvements on my tests, I think it still provide some different insights into how each library handles heavy loads and concurrency.
Links:
You can read my findings on my blog
Check out the source code: on Github
Thanks
/r/Python
https://redd.it/1digyfg
What My Project Does
While looking for task queues, I found that there are many options available in the Python ecosystem, making it really hard to choose the right one. To get a sense of how each library performs and to help make an informed decision, I conducted a load test on some of the most popular ones: Python-RQ, ARQ, Celery, Huey, and Dramatiq.
Target Audience
I hope my findings can help those who are also looking for a task queue solution in Python.
Comparison
Most articles out there seem to focus on comparing the features of these libraries but rarely discuss performance. While there could be a lot of improvements on my tests, I think it still provide some different insights into how each library handles heavy loads and concurrency.
Links:
You can read my findings on my blog
Check out the source code: on Github
Thanks
/r/Python
https://redd.it/1digyfg
Tech Notes from Steven
Exploring Python Task Queue Libraries with Load Test
Task queues play a crucial role in managing and processing background jobs efficiently. Whether you need to handle asynchronous tasks, schedule periodic jobs, or manage a large volume of short-lived tasks, task queue libraries can help you achieve these goals.
Is there a package that simplifies the CRUD process in django
I've been using Django for the past three years and one of the parts that if l could simplify l would be very happy will be the CRUD process because it could save a lot of time. Imagine having 10 models and you need to create a CRUD process for all of them. I hope you feel me. So if there is, fine, if not, then l will start working on something like that, if you are interested too, you let me know. Thanks.
/r/django
https://redd.it/1dilbip
I've been using Django for the past three years and one of the parts that if l could simplify l would be very happy will be the CRUD process because it could save a lot of time. Imagine having 10 models and you need to create a CRUD process for all of them. I hope you feel me. So if there is, fine, if not, then l will start working on something like that, if you are interested too, you let me know. Thanks.
/r/django
https://redd.it/1dilbip
Reddit
From the django community on Reddit
Explore this post and more from the django community
rug 0.10.4 released
What's rug library:
Library for fetching various stock data from the internet (official and unofficial APIs).
Source code:
https://gitlab.com/imn1/rug
Releases including changelog:
https://gitlab.com/imn1/rug/-/releases
/r/Python
https://redd.it/1dilsv3
What's rug library:
Library for fetching various stock data from the internet (official and unofficial APIs).
Source code:
https://gitlab.com/imn1/rug
Releases including changelog:
https://gitlab.com/imn1/rug/-/releases
/r/Python
https://redd.it/1dilsv3
GitLab
n1 / rug · GitLab
Library for fetching various stock data from the internet (official and unofficial APIs).
I just made my first API wrapper: beachwatch
Source code: https://github.com/ben-n93/beachwatch
What my Project Does
In NSW, Australia the government provides an API with daily updates on beaches' pollution and water quality forecast.
I thought I'd make a simple wrapper in Python to make it easier to get data from the API.
Target Audience
Most likely Australian data scientists/analysts/developers interested in beach data.
/r/Python
https://redd.it/1diovo8
Source code: https://github.com/ben-n93/beachwatch
What my Project Does
In NSW, Australia the government provides an API with daily updates on beaches' pollution and water quality forecast.
I thought I'd make a simple wrapper in Python to make it easier to get data from the API.
Target Audience
Most likely Australian data scientists/analysts/developers interested in beach data.
/r/Python
https://redd.it/1diovo8
GitHub
GitHub - ben-n93/beachwatch: A Python wrapper for the NSW Government's Beachwatch API.
A Python wrapper for the NSW Government's Beachwatch API. - ben-n93/beachwatch
Scrapegraph AI Tutorial; Scrape Websites Easily With LLaMA AI
I'm going to show you how to get Scrapegraph AI up and running, how to set up a language model, how to process JSON, scrape websites, use different AI models, and even turning your data into audio. Sounds like a lot, but it's easier than you think, and I'll walk you through it step by step.
https://www.scrapingbee.com/blog/scrapegraph-ai-tutorial-scrape-websites-easily-with-llama-ai/
/r/Python
https://redd.it/1dip146
I'm going to show you how to get Scrapegraph AI up and running, how to set up a language model, how to process JSON, scrape websites, use different AI models, and even turning your data into audio. Sounds like a lot, but it's easier than you think, and I'll walk you through it step by step.
https://www.scrapingbee.com/blog/scrapegraph-ai-tutorial-scrape-websites-easily-with-llama-ai/
/r/Python
https://redd.it/1dip146
Scrapingbee
Scrapegraph AI Tutorial; Scrape websites easily with LLaMA AI | ScrapingBee
Explore how Scrapegraph AI simplifies web scraping using AI, making it easy to extract data from dynamically changing websites. Learn to set up the tool.
Website with dynamic pages
For something like a discussion website (like reddit).
When the user makes a post. whats the best way to create the page.
Right now, i have 1 html file in the statics folder. It works as a template for all posts. When a user clicks on a specific post. It populates that template with all the respective data for that post.
The only thing im thinking is since there is no specific page for each post. lets say someone was to search directly for a post using its title through google. they wouldnt find it since there is no designated html page.
Im thinking. would it be a good idea to change the way the posts are made and instead - make a specific html page for each post. This will end up taking up a lot more space in the static folder as more and more html pages are added.
/r/flask
https://redd.it/1ditod2
For something like a discussion website (like reddit).
When the user makes a post. whats the best way to create the page.
Right now, i have 1 html file in the statics folder. It works as a template for all posts. When a user clicks on a specific post. It populates that template with all the respective data for that post.
The only thing im thinking is since there is no specific page for each post. lets say someone was to search directly for a post using its title through google. they wouldnt find it since there is no designated html page.
Im thinking. would it be a good idea to change the way the posts are made and instead - make a specific html page for each post. This will end up taking up a lot more space in the static folder as more and more html pages are added.
/r/flask
https://redd.it/1ditod2
Reddit
From the flask community on Reddit
Explore this post and more from the flask community
u really need to try this admin panel
so i found this django admin panel , it is amazing
https://unfoldadmin.com/
/r/djangolearning
https://redd.it/1dgkulo
so i found this django admin panel , it is amazing
https://unfoldadmin.com/
/r/djangolearning
https://redd.it/1dgkulo
Unfold
Unfold - Admin theme for Django
Modern Django admin theme for seamless interface development
Wednesday Daily Thread: Beginner questions
# Weekly Thread: Beginner Questions 🐍
Welcome to our Beginner Questions thread! Whether you're new to Python or just looking to clarify some basics, this is the thread for you.
## How it Works:
1. Ask Anything: Feel free to ask any Python-related question. There are no bad questions here!
2. Community Support: Get answers and advice from the community.
3. Resource Sharing: Discover tutorials, articles, and beginner-friendly resources.
## Guidelines:
This thread is specifically for beginner questions. For more advanced queries, check out our [Advanced Questions Thread](#advanced-questions-thread-link).
## Recommended Resources:
If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance.
## Example Questions:
1. What is the difference between a list and a tuple?
2. How do I read a CSV file in Python?
3. What are Python decorators and how do I use them?
4. How do I install a Python package using pip?
5. What is a virtual environment and why should I use one?
Let's help each other learn Python! 🌟
/r/Python
https://redd.it/1dj5f2u
# Weekly Thread: Beginner Questions 🐍
Welcome to our Beginner Questions thread! Whether you're new to Python or just looking to clarify some basics, this is the thread for you.
## How it Works:
1. Ask Anything: Feel free to ask any Python-related question. There are no bad questions here!
2. Community Support: Get answers and advice from the community.
3. Resource Sharing: Discover tutorials, articles, and beginner-friendly resources.
## Guidelines:
This thread is specifically for beginner questions. For more advanced queries, check out our [Advanced Questions Thread](#advanced-questions-thread-link).
## Recommended Resources:
If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance.
## Example Questions:
1. What is the difference between a list and a tuple?
2. How do I read a CSV file in Python?
3. What are Python decorators and how do I use them?
4. How do I install a Python package using pip?
5. What is a virtual environment and why should I use one?
Let's help each other learn Python! 🌟
/r/Python
https://redd.it/1dj5f2u
Discord
Join the Python Discord Server!
We're a large community focused around the Python programming language. We believe that anyone can learn to code. | 390070 members
Flask App Deployment w/ SocketIo, SQLAlchemy, Requests + BS4
Hi, i'm very new to deployment and have made a flask app that works well locally. It has a chatroom function that uses flask socketio and a database visualiser that uses requests + bs4 to populate a sqlalchemy database from a website.
I'm trying to find a deployment platform that can handle all of this but i'm getting issues.
On pythonanywhere, the database pulling + displaying works through requests, but the socketio doesn't work for the chat function.
On heroku, neither of them work, but the databasing works only for user account creation.
What recommendations do people have for the best way to deploy this? I'm happy to provide my code also.
/r/flask
https://redd.it/1dj7f6h
Hi, i'm very new to deployment and have made a flask app that works well locally. It has a chatroom function that uses flask socketio and a database visualiser that uses requests + bs4 to populate a sqlalchemy database from a website.
I'm trying to find a deployment platform that can handle all of this but i'm getting issues.
On pythonanywhere, the database pulling + displaying works through requests, but the socketio doesn't work for the chat function.
On heroku, neither of them work, but the databasing works only for user account creation.
What recommendations do people have for the best way to deploy this? I'm happy to provide my code also.
/r/flask
https://redd.it/1dj7f6h
Reddit
From the flask community on Reddit
Explore this post and more from the flask community
Parsing Python ASTs 20x faster with Rust
https://www.gauge.sh/blog/parsing-python-asts-20x-faster-with-rust
/r/Python
https://redd.it/1div2e8
https://www.gauge.sh/blog/parsing-python-asts-20x-faster-with-rust
/r/Python
https://redd.it/1div2e8
www.gauge.sh
Parsing Python ASTs 20x Faster with Rust
When CPython Is Too Slow
A JIT compiler for CPython
Brandt Bucher talks on JIT compiler for Python at CPython Core Developer Sprint. Brandt is a member of the Faster CPython project, which is working on making the reference implementation of the language faster via a variety of techniques.
https://www.youtube.com/watch?v=HxSHIpEQRjs
/r/Python
https://redd.it/1djdy0z
Brandt Bucher talks on JIT compiler for Python at CPython Core Developer Sprint. Brandt is a member of the Faster CPython project, which is working on making the reference implementation of the language faster via a variety of techniques.
https://www.youtube.com/watch?v=HxSHIpEQRjs
/r/Python
https://redd.it/1djdy0z
GitHub
faster-cpython
faster-cpython has 6 repositories available. Follow their code on GitHub.
The worst Python-Django backend refactoring mission of all time?
Are you looking to refactor some horrible messy code?
Let's do it together. 🎉
Evaboot in few words:
remote-only company.
bootstrapped.
2M€ ARR in 2.5 years with 2 people.
We are building software to help sales teams create prospecting databases from Linkedin.
Here is a demo of the tool:
[https://www.youtube.com/watch?v=khYgyDlo2Iw](https://www.youtube.com/watch?v=khYgyDlo2Iw)
# Roadmap of the job
→ For the next 2 months, you will refactor the Python-Django backend code of Evaboot. It will not be a rebuild, but a continuous refactoring of the existing codebase.
→ After that, you'll dive right into new features development: we got many pain points to solve for our users ❤️🩹.
❓How: You will work directly with [Robin](https://www.linkedin.com/in/zrobin/)
❓Where: anywhere on the planet — we are remote since day one.
# Requirements
→ Serious experience in Python, Django & Celery.
→ You have eaten a lot of spaghetti code before 🍝 (and not just your own).
# Bonus points
→ you worked with the Heroku Cloud ecosystem.
→ you worked with Django REST framework.
→ you worked on scraping projects.
# What's guaranteed
→ Intense execution pace & exponential learning curve.
→ Zero bullshit environment. We focus mostly on results and KPIs.
→ No wasted code. We take user pain seriously, code is here to solve it.
# How To Apply
\*Apply right here:** https://docs.google.com/forms/d/e/1FAIpQLScdHbxUA64x7L7dkpWtC\_aT5XI9hSUBw2uhaUka7detFJFUPQ/viewform
Don’t be shy —
/r/django
https://redd.it/1djcbu5
Are you looking to refactor some horrible messy code?
Let's do it together. 🎉
Evaboot in few words:
remote-only company.
bootstrapped.
2M€ ARR in 2.5 years with 2 people.
We are building software to help sales teams create prospecting databases from Linkedin.
Here is a demo of the tool:
[https://www.youtube.com/watch?v=khYgyDlo2Iw](https://www.youtube.com/watch?v=khYgyDlo2Iw)
# Roadmap of the job
→ For the next 2 months, you will refactor the Python-Django backend code of Evaboot. It will not be a rebuild, but a continuous refactoring of the existing codebase.
→ After that, you'll dive right into new features development: we got many pain points to solve for our users ❤️🩹.
❓How: You will work directly with [Robin](https://www.linkedin.com/in/zrobin/)
❓Where: anywhere on the planet — we are remote since day one.
# Requirements
→ Serious experience in Python, Django & Celery.
→ You have eaten a lot of spaghetti code before 🍝 (and not just your own).
# Bonus points
→ you worked with the Heroku Cloud ecosystem.
→ you worked with Django REST framework.
→ you worked on scraping projects.
# What's guaranteed
→ Intense execution pace & exponential learning curve.
→ Zero bullshit environment. We focus mostly on results and KPIs.
→ No wasted code. We take user pain seriously, code is here to solve it.
# How To Apply
\*Apply right here:** https://docs.google.com/forms/d/e/1FAIpQLScdHbxUA64x7L7dkpWtC\_aT5XI9hSUBw2uhaUka7detFJFUPQ/viewform
Don’t be shy —
/r/django
https://redd.it/1djcbu5
Evaboot
Linkedin Sales Navigator Scraper - Evaboot Chrome Extension
The Smartest Linkedin Sales Navigator Scraper. Our Linkedin Sales Navigator Extractor clean, extract and enrich all Sales Navigator search results.
I'm looking for a comparison with a Python developer ITA.
I am a high school student and I wanted to look for someone who works as a Python developer from Italy, I don't know where else I can look for some Italian developers, if you know of places where I can get some advice I thank you.
/r/Python
https://redd.it/1djh9ri
I am a high school student and I wanted to look for someone who works as a Python developer from Italy, I don't know where else I can look for some Italian developers, if you know of places where I can get some advice I thank you.
/r/Python
https://redd.it/1djh9ri
Reddit
From the Python community on Reddit
Explore this post and more from the Python community
Looking for a mentor or project
Hi everyone I’m a recent graduate and I’ve been looking for an internship for a while with no luck.
I’ve built my own projects but I’d love to have a reference on my resume to a person.
If there’s anyone out there that can let me help them out with their SAAS or small project they have I would work for free and be more than grateful.
/r/django
https://redd.it/1djjmx5
Hi everyone I’m a recent graduate and I’ve been looking for an internship for a while with no luck.
I’ve built my own projects but I’d love to have a reference on my resume to a person.
If there’s anyone out there that can let me help them out with their SAAS or small project they have I would work for free and be more than grateful.
/r/django
https://redd.it/1djjmx5
Reddit
From the django community on Reddit
Explore this post and more from the django community
Textual Serve - Serve TUIs in the browser
Textual Serve (https://github.com/Textualize/textual-serve) is a project which serves TUIs (built with Textual) in the browser.
This is self-hosted, so you don't need to rely on any external service.
/r/Python
https://redd.it/1djk6u8
Textual Serve (https://github.com/Textualize/textual-serve) is a project which serves TUIs (built with Textual) in the browser.
This is self-hosted, so you don't need to rely on any external service.
/r/Python
https://redd.it/1djk6u8
GitHub
GitHub - Textualize/textual-serve: Serve Textual apps locally
Serve Textual apps locally. Contribute to Textualize/textual-serve development by creating an account on GitHub.
[P] [D] Updated On: Hi I'm a senior machine learning engineer, looking for for buddies to build cool stuff with!
**Old Post**: [https://www.reddit.com/r/MachineLearning/comments/1dj8pg6/p\_d\_hi\_im\_a\_senior\_machine\_learning\_engineer/](https://www.reddit.com/r/MachineLearning/comments/1dj8pg6/p_d_hi_im_a_senior_machine_learning_engineer/)
**Wow, I wasn't expecting this post to gain so much attention!**
I've created a Google form for those who are serious about working on projects, Kaggle competitions, or Leetcode challenges. Please take a moment to fill it out.
Form: [https://forms.gle/k3jzCfNJy3rgz4ec6](https://forms.gle/k3jzCfNJy3rgz4ec6)
Here's how it will work:
* I will create groups based on your goals and expertise.
* Each group will have a team leader to assist with progress, alongside my support.
* It will take some time to organize everyone into teams, so please be patient. I'll reach out to you soon.
Thank you!
/r/MachineLearning
https://redd.it/1djmpwx
**Old Post**: [https://www.reddit.com/r/MachineLearning/comments/1dj8pg6/p\_d\_hi\_im\_a\_senior\_machine\_learning\_engineer/](https://www.reddit.com/r/MachineLearning/comments/1dj8pg6/p_d_hi_im_a_senior_machine_learning_engineer/)
**Wow, I wasn't expecting this post to gain so much attention!**
I've created a Google form for those who are serious about working on projects, Kaggle competitions, or Leetcode challenges. Please take a moment to fill it out.
Form: [https://forms.gle/k3jzCfNJy3rgz4ec6](https://forms.gle/k3jzCfNJy3rgz4ec6)
Here's how it will work:
* I will create groups based on your goals and expertise.
* Each group will have a team leader to assist with progress, alongside my support.
* It will take some time to organize everyone into teams, so please be patient. I'll reach out to you soon.
Thank you!
/r/MachineLearning
https://redd.it/1djmpwx
Reddit
From the MachineLearning community on Reddit
Explore this post and more from the MachineLearning community
N Ilya Sutskever and friends launch Safe Superintelligence Inc.
With offices in Palo Alto and Tel Aviv, the company will be concerned with just building ASI. No product cycles.
https://ssi.inc
/r/MachineLearning
https://redd.it/1djrs3n
With offices in Palo Alto and Tel Aviv, the company will be concerned with just building ASI. No product cycles.
https://ssi.inc
/r/MachineLearning
https://redd.it/1djrs3n
ssi.inc
Safe Superintelligence Inc.
The world's first straight-shot SSI lab, with one goal and one product: a safe superintelligence.
Python Project Management Primer
This article explores how to manage Python project environments and dependencies, as well as how to structure projects effectively.
/r/Python
https://redd.it/1djp1gw
This article explores how to manage Python project environments and dependencies, as well as how to structure projects effectively.
/r/Python
https://redd.it/1djp1gw
MLOps Shenanigans
Python Project Management Primer
Alleviating Python Developer Pain
Friday Daily Thread: r/Python Meta and Free-Talk Fridays
# Weekly Thread: Meta Discussions and Free Talk Friday 🎙️
Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related!
## How it Works:
1. Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community.
2. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community.
3. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting.
## Guidelines:
All topics should be related to Python or the /r/python community.
Be respectful and follow Reddit's Code of Conduct.
## Example Topics:
1. New Python Release: What do you think about the new features in Python 3.11?
2. Community Events: Any Python meetups or webinars coming up?
3. Learning Resources: Found a great Python tutorial? Share it here!
4. Job Market: How has Python impacted your career?
5. Hot Takes: Got a controversial Python opinion? Let's hear it!
6. Community Ideas: Something you'd like to see us do? tell us.
Let's keep the conversation going. Happy discussing! 🌟
/r/Python
https://redd.it/1dkqdsg
# Weekly Thread: Meta Discussions and Free Talk Friday 🎙️
Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related!
## How it Works:
1. Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community.
2. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community.
3. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting.
## Guidelines:
All topics should be related to Python or the /r/python community.
Be respectful and follow Reddit's Code of Conduct.
## Example Topics:
1. New Python Release: What do you think about the new features in Python 3.11?
2. Community Events: Any Python meetups or webinars coming up?
3. Learning Resources: Found a great Python tutorial? Share it here!
4. Job Market: How has Python impacted your career?
5. Hot Takes: Got a controversial Python opinion? Let's hear it!
6. Community Ideas: Something you'd like to see us do? tell us.
Let's keep the conversation going. Happy discussing! 🌟
/r/Python
https://redd.it/1dkqdsg
Redditinc
Content Policy
How do you organize your flask projects?
Do you mimic the structure of a django project? Something else? Would like to hear what other people here do as I am pretty used to opinionated frameworks for web like aforementioned django and spring (boot) and feel a little disorganized with how I have my projects right now.
/r/flask
https://redd.it/1dkoh6r
Do you mimic the structure of a django project? Something else? Would like to hear what other people here do as I am pretty used to opinionated frameworks for web like aforementioned django and spring (boot) and feel a little disorganized with how I have my projects right now.
/r/flask
https://redd.it/1dkoh6r
Reddit
From the flask community on Reddit
Explore this post and more from the flask community