Feature Requests
Jes Auto Count (with AI)
Was curious how people feel about an auto count feature down the line with AI. I have a pretty bad taste in my mouth using this before in other software/platforms.
The premise is simple. Select an item on a page and then instruct the AI to attempt to find and count all the matches for you.
When I've used this feature elsewhere, most of the time the OCR/AI/whatever either misses things or counts things that shouldn't be counted. Sometimes it performs well but you never know.
I know the accuracy in other software was heavily dependent on the image quality of the floor plans, and whether or not the PDFs have text layers embedded.
In the example above, if the PDF does not have text layers, some of the text above will get missed in other software due to lines crossing through them, or the text is running vertically. C-3 in the image above would be missed because a line is crossing through the 3.
I would imagine people would want functionality like this, but only if the inherent challenges can be overcome
I haven't seen it work effectively - have tried it on OST and Togal - i just don't trust it for counts or linear. It's decent at Area. But if you asked me what i'd want out of that screenshot, it would be to click on that section tag and have it take me straight to that, or detail. (bluebeam has this)
I think it is a game changer while it is working…but the second it does anything wrong it loses all confidence. So no, I’m not very excited about it.
I’ve always assumed with AI counters that the counts have to be checked by a human (I find it reassuring that I won’t be replaced by AI… yet). I found the Esticom model interesting where the AI would give you a pop up where it grouped its findings into categories of things it’s pretty sure it should count and things it’s not sure of (due to intersecting lines or other drawing noise). You can then select or deselect any of those findings before it finalizes its count. It’s still faster than clicking on everything and more accurate (and faster) than having to scour the drawing to check the AI’s work.
When using AI to help us with coding, it is wrong a lot. The good news is that even though it's wrong, we can sift through it's suggestions (like a big pile of scrap metal you can use to build something) and find pieces to assemble together into something useful. In the end it saves us time, but we have to be very cautious to review what it gives us. With this in mind, our approach with AI is to keep the estimator in the drivers seat, and ensure that AI is primarily working as an assistant to reduce or eliminate redundant tasks.
We do intend to have AI scan the plans and be helpful finding things, but our general approach would be to present the information to the estimator for validation in a way that is as helpful as possible and easy to verify. Making the verification process simple and easy is the key.
Just my personal opinion on Apple / iPhone, but the most useful update that I personally experienced from iPhone in the last 5 years is accuracy of Siri. At first, I had to correct its mistakes so many times that it was faster to just type (dumped Siri). Now I use it constantly for reminders or notes and I rarely have to correct it. I think AI with takeoffs will take a similar path. Right now it requires a lot of hand holding, and the key is making it easy, but not trusting it too much. We do intend to have an auto count feature, but how it presents the results and allows the estimator to verify will be super important. We could also make it work in the background to find and present possible insights / hints..."you counted all these fixtures, but did you know there was another one that is on the Foundation Plan that didn't get counted". If you intentionally skipped the fixture, the warning is no problem, but if it helps you catch one you missed, that's great.
Down the road we will hopefully develop more advanced models that can be much more accurate and useful in helping auto count, build templates / formulas / assemblies / plugins, uncovering insights on possible helpful hints, voice control, auto takeoff walls & areas, auto scale, etc. Lots of fun stuff. 🙂
Wanted to give a real world example.
Below are windows labeled on a floor plan, and part of the window schedule providing the width & height of the windows:
There are probably 100 of these on the floor I'm currently working on.
It would be awesome if I could have AI run through the floor using that schedule and mark these spots with the window heights, even if it was merely a hover detail it could apply when I move my mouse over the label. Would save me a bunch of back and forth between looking at the plans and the schedule while I'm doing some linear takeoffs.
I know this goes beyond a basic auto count function, it's more like search & label, with the goal of assisting the estimator, not necessarily doing the work for them.
Thanks for sharing Jes. I love these real world examples for us to discuss.
What about on DFH and Landscape Schedules? That should be much more straight forward?