Doc-Pi

Tell us about your projects. Update us regularly.
parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Doc-Pi

Post by parkview » Wed Jul 09, 2014 8:15 pm

One project I have on the go, is a Raspberry Pi controlled document photographer. The rig will allow me to photograph 6"x4" photographs and up to A4 documents. There will be a glass lid that then closed, will trigger two 18W bar LEDs to turn on, and the RPi camera to take a photo. The LEDs will then turn off, so I can then lift the glass lid and swap out the document. I am hoping I can quickly photograph a lot of documents quite quickly. If nothing else, it's been a fun project.

I have all the code written up, and am now working on the hardware:

example_glass_holder.jpg
example_glass_holder.jpg (24.29 KiB) Viewed 17295 times


I had two of these lying around: http://www.dx.com/p/hzled-10w-1050lm-60 ... 70xf0Dwtek but they turned out not to be long enough, so I have now ordered two x 20cm long LEDs: http://www.dx.com/p/18w-1710-1890lm-650 ... 70xk0Dwtek I hope they aren't too bright! :)

User avatar
Jubbp
Master Maker
Posts: 209
Joined: Sun Jun 22, 2014 8:15 pm
Location: Bunbury WA
Contact:

Re: Doc-Pi

Post by Jubbp » Tue Jul 15, 2014 11:02 am

Did you draw up the parts?

What was the CAD app you use again?

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Tue Jul 15, 2014 10:05 pm

Jubbp wrote:Did you draw up the parts? What was the CAD app you use again?


Yes. I use the free DesignSpark Mechanical: http://www.rs-online.com/designspark/el ... mechanical Seems to work for me. First up it appeared to have a few quirks, but if you actually RTFM, it mostly covers off on things. It's now a joy to use. I have trialled FreeCAD, but my PC had issues with ver 13 and 14, in that it would just crash, sometimes a lot. BeJay swears by it though, so perhaps it was just my PC, or the operator :|

I have printed a few pieces of it to make sure the parts fit together. When finished, it will be 1cm larger than an A4 sheet of paper. I pick up some 3mm and 2mm thick glass for it this Sat. I am currently working on the back hinge area, but am stuck on sides, as I need the new 20cm LED bar to arrive, so I can figure out where to mount them.

Stephen, do you still use your paid software?

User avatar
BeJay
Maker
Posts: 139
Joined: Mon Jun 23, 2014 6:31 pm
Location: Perth
Contact:

Re: Doc-Pi

Post by BeJay » Fri Jul 18, 2014 8:13 pm

Yes I do love FeeCAD, it works fine on LinuxMint14 and now 17. I'm using version 0.14 rev 3618 GIT. The only issues I've really had are with text. FreeCAD requires you to draft text and import it into the "PART" section wich can be fiddly. I use inkscape for text and import it into FreeCAD instead.

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Sun Jun 07, 2015 7:51 pm

The Quest for the Perfect LED Arm: Part 1

This quest started back in 2014, just before my 3D printer bit the dust. Back then my initial LED arm was just an upright bar. My thinking was that as long as the LED bar was placed at a 45 deg. angle, then the camera should be fine. Wrong! Obviously it wasn't until version three, that I really did some serious trials and measurements..

Here is version 2 (LH side) and Ver. 1 (RH side) of the 10W LED brackets/arms:

doc-pi - led holders 2 and 1.jpg
doc-pi - led holders 2 and 1.jpg (42.85 KiB) Viewed 17280 times


I kind of liked version 2 because of the fun I had with playing around with splines. The LED bar angle was a bit low though. The bread board off to the right is fully set for the activation via a micro switch of the LED lights. You close the assembly, the lights come on and the camera automatic tales a photo and the LEDs go off letting you know you can load a new document.
Last edited by parkview on Sun Jun 07, 2015 8:05 pm, edited 1 time in total.

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Sun Jun 07, 2015 8:04 pm

The Quest for the Perfect LED Arm: Part 2

With the Felix 3D printer back in action, it's been a busy weekend. I have designed and printed another three iterations of the DOC-Pi LED bracket

Ver. 3: RH side, Moving the LED light bar back away from the glass, so there is no/less LED reflection (Ver. 4 on LH side):
doc-pi - led holders V3 and V2.jpg
doc-pi - led holders V3 and V2.jpg (28.56 KiB) Viewed 17280 times


Ver. 4: Still have reflections, move the LED bar array back even further. This however, introduced shadows from the supports:
doc-pi - led holders V3 shadows.jpg
doc-pi - led holders V3 shadows.jpg (28.22 KiB) Viewed 17280 times


Ver. 5 & 6: Now the the LED array 85mm off tot he left/right and ~130mm up from the glass platen. Just a tiny shadow to get rid off. Left hand LED brackets are no. 5 and the right hand is the 6th and final iteration:
doc-pi - led holders V5 and V6.jpg
doc-pi - led holders V5 and V6.jpg (21.84 KiB) Viewed 17280 times


IN a few weeks time, I will work on designing up a mount for the RPi camera. This will sit on a 3D printed frame that's attached to the LED brackets. I also need to cut out a 6mm thick MDF base and mount the hinges and micro-switch to it.

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Sun Jul 05, 2015 10:02 pm

Had fun on the weekend designing up a camera sled and camera mount for DOC-Pi.

The requirement was that it had to be adjustable, so I could slide the camera up and down a bit. I thought I would start with this, so I can fine tune the location while I am testing the unit.

It's built in two pieces, one being the camera mount, which is based on the first thing I 3D designed. The second is the mount for the camera mount. It wasn't till I had printed it out and held it in my hand that I realised it was a lot more complicated that it needed to be. I could just have used the camera nylon bolts to attach the camera mount to the camera sled. Oh well. The 2mm nylon bolts I use are not that robust. At least the metal bolts will out last the sled unit. I am not so sure about the nylon bolts.

Now I have to design up the four arms that will link the sled tot he LED assemblies. Not too sure about this, as I didn't really think this all through. It's been another iterative project. At least by the end, I will know how to design a second prototype much faster and skipping all the errors I have previously made.

Camera sled assembly.jpg
Camera sled assembly.jpg (25.28 KiB) Viewed 17275 times


While in Shenzhen, I found some neat 8mm wide x 100mm long, 5W cool white bar LED's. There is a special LED building that's 5 floors high and has three floors dedicated to LEDs of all type of power, colour and size. It's an astounding place. It's full of bedroom sized stalls, fronted by glass cabinets:

just a few LEDs.jpg
just a few LEDs.jpg (45.92 KiB) Viewed 17275 times


I did spend a bit on some different shapes and sized LEDs. Watch out for some future projects using these. Anyway, I now had a tray of the 8mm x 100mm long bar LEDs. No the stall holder doesn't have any aluminium heat sinks for them, so off I wondered looking for some. Special sized LEDs meant that there was only one stall that sold it. It took 40min. to find it some two floors below, and they where large floors! It was a fun place to wonder around in though.

The guy couldn't speak English, and I only knew two words in Chinese, but with the help of paper hand drawings and an English to Chinese App, I ordered 2 x 400mm long strips of it for a few dollars. He said they would be ready the following day, and they where. Here is the finished 15W LED assembly after they have been soldered together:

DOC-Pi  - new LED assemblys.jpg
DOC-Pi - new LED assemblys.jpg (12.62 KiB) Viewed 17275 times


By the by, the in thing over there at the moment is LED panels. These can be either round, square or rectangular and be 12cm up to 100+cm in size. You can also pick up plastic opaque 3D images. The idea is that you buy a stack of different ones, and they sit over the LED panel and the images appears to change and move as you walk around the room. When you get bored, you just peel it off and replace it with another one.

:idea: I wonder how long before the entire ceiling is covered in LED's. One could use a smart phone App to turn on the LED's you want lit and the intensity. Only you in the room? - fine, just turn on the LED's above where you are standing or sitting. LOL - or you could have a M$ Kinetic style device that automatic knows where you are in the room and it just turns on the LEDs above you - as you walk around. :idea:

User avatar
seaton
Master Maker
Posts: 222
Joined: Tue Jun 24, 2014 11:41 am
Location: Bunbury, WA
Contact:

Re: Doc-Pi

Post by seaton » Wed Jul 08, 2015 12:40 pm

you have been busy....nice one

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Sat Sep 19, 2015 5:45 pm

After a bit of a hiatus, I have spent some time working on the Doc-Pi project. I have now added a two layered base, and hinges for the 3D Printed frame to sit on. This means I now have a number of screws and bolts protruding underneath the base. I pondered how I was going to fit some 15mm deep feet, without resorting to purchasing something from Bunnings.

In a flash, I thought up a two part printed PLA foot. One to hold the head of the bolt from pulling through the soft MDF board and then, actual foot underneath. I had some Ninjaflex lying around that I hadn't used before, so I printed out some rubber feet for them as well.

As per usual, it's all designed up in free Designspark Mechanical software. Here is a cut away of all the pieces (no nut on the bolt):

MDF Feet - design-annotated.jpg
MDF Feet - design-annotated.jpg (174.48 KiB) Viewed 17268 times


The photo below, shows the countersunk (using a spade drill), top of the foot assembly. You can also see a slightly skewed micro switch assembly. It was only when I tried countersinking the three microswitch bolts that I realised how easily the bolt heads would pull through the MDF. The 3D printed countersunk feet was the fix for that issue:

Top of Foot.jpg
Top of Foot.jpg (144 KiB) Viewed 17268 times

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Sat Sep 19, 2015 6:08 pm

Apart from mounting the RPi Camera up on top of the LED floodlights, the rest of the base is pretty much completed. It's time to work on the electronics. I had worked on it back in July 2014, but the original breadboard had been subsequently subsumed into two other projects, so I dug out a fresh smaller breadboard and rebuilt the electronics:

Doc-Pi breadboarded electronics.jpg
Doc-Pi breadboarded electronics.jpg (182.47 KiB) Viewed 17267 times


for anyone interested in the simple schematics of the above circuitry, I have hand drawn them out here:

Doc-Pi schematics.jpg
Doc-Pi schematics.jpg (120.39 KiB) Viewed 17267 times


Lastly, here is an action shot of it all. I am using the newly released RPi Display to preview what the camera sees:

Testing Doc-Pi.jpg
Testing Doc-Pi.jpg (224.99 KiB) Viewed 17267 times


The code is pretty rudimentary at the moment. When the lid closes on the microswitch, the RPi turns on the floodlight LEDs and has the camera take a photo. The images are all date and time coded and filed away in a sub-directory and a master file list of the images is kept as well. If I stick with the RPi display, I could easily write up a simple touch screen Pygame front end for it all, but maybe a $92 display might be a bit too much overhead for such a simple project?

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Sat Sep 19, 2015 6:42 pm

RPi Display Footnote:

My initial attempts to use the RPi 2B being powered by the RPi 7" Display AND the RPi camera failed!

No matter how I tried taking a photo via the RPi camera, the preview would appear on the screen and the camera red LED would light up and a blank file would be created, but that was it. A photo never appeared and I would have to kill the process!

It turned out that even with a 10W 5V PSU, my USB cable dropped too much voltage and while it was enough to run both the RPi and the Display, it couldn't put out another 200mA at 5V to properly run the camera. The voltage drooped and that was that, game over.

I guess, if I keep with the official RPi Display, I might put in two separate 12V (I need this to run the LED document flood lights), to 5V @ 1A converters to run each device separately.

User avatar
Jubbp
Master Maker
Posts: 209
Joined: Sun Jun 22, 2014 8:15 pm
Location: Bunbury WA
Contact:

Re: Doc-Pi

Post by Jubbp » Wed Sep 23, 2015 9:26 am

This project is coming along nicely. ** Thumbs Up **

Might need to actually get around to trying DS Mechanical...

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Sun Oct 04, 2015 10:05 pm

Well, the Felix 3D Printer has been running hot this weekend, busy printing out a newly designed LED support arm to fit the 3 x 5W 8mm bar LED's and - finally - the last missing piece of the DOC-Pi assembly, a bracket that runs from the top of the LED arm up to the top camera sled (brown - only two shown):

new_camera_brackets.jpg
new_camera_brackets.jpg (26 KiB) Viewed 17281 times


I love the accuracy of it all. If you design it up well and you test it all out in the 3D-CAD program, then you just have to print it all out and it fits/bolts/snaps together perfectly!

So, how does it all run? Well so far, quite nicely. Here is a overall photo of it in action:

doc-pi_overhead.jpg
doc-pi_overhead.jpg (68.6 KiB) Viewed 17281 times


and the resultant photo from the overhead RPi Camera. The photos come out much clearer than I thought they might. Of course, it looks much nicer at full resolution:

2015-10-04_212413.jpg
2015-10-04_212413.jpg (71.8 KiB) Viewed 17281 times


* I am impressed that the tiny 12V relay can drive the 2 x 15W LED flood lights with out melting down. I was expecting that I might have to use one for each 15W flood light.

* At the moment, there is a CLI Pygame RPi 7" display/touchscreen mouse driver issue, in that while the touchscreen is fully calibrated within xWindows environment, in a CLI Pygame environment, it's way out. I guess I could ruin the DOC-Pi program via xWindows, but I do like operating in a pure command-line-interface.

* You can see in the overhead photo that I have mounted the camera 90deg. out, so it's great at taking side to side photos, but not the length of the platen. I will need to re-design the small camera sled and re-print it.

User avatar
BeJay
Maker
Posts: 139
Joined: Mon Jun 23, 2014 6:31 pm
Location: Perth
Contact:

Re: Doc-Pi

Post by BeJay » Sun Oct 04, 2015 11:36 pm

That is tidy! Nice work.

For a 30W load, just use a MTP3055 or equiv MOSFET. It won't even get hot at that load ;)

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Mon Oct 05, 2015 7:41 pm

Thanks BeJay.

I have many fond memories of playing around with 2N3055 transistors in the 80's and 90's. Due to the heat load, my Hi-Voltage rig can only run for 30-45 seconds using a heatsinked 2N3055 transistor. You have reminded me of the project. I think it's sitting in the shed in pieces at the moment. I can see a few blog posts coming at some stage down the road.

I see they do a TTL version of the MOSFET too: MTP3055VL

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Wed Oct 07, 2015 8:25 pm

The MPT3055VL MOSFETS arrived today from Altronics in Perth. They work a treat. I can see them being used in a lot of places. Thanks for the tip BeJay!

User avatar
Jubbp
Master Maker
Posts: 209
Joined: Sun Jun 22, 2014 8:15 pm
Location: Bunbury WA
Contact:

Re: Doc-Pi

Post by Jubbp » Mon Oct 12, 2015 10:25 am

It's been great watching this project from the beginning until now. Great work Paul. :) :)

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Mon Nov 16, 2015 9:28 pm

I have extended the camera mounting brackets by an extra 40mm, so I now can photograph a full A4 sheet of paper. The RPi is now mounting a NFS share on my home QNAP, so no data files are now stored on the Doc-Pi.

This is now turning more into a EDMS (Electronic Document Management System) and software project. I do have to convert the breadboard into a permanent circuit board, but the rest is kind of software.

I have lots of ideas on this and have pondered using the community version of EDMS packages like Knowledge Tree (https://www.knowledgetree.com/) or Alfresco (https://www.alfresco.com/community). After a quick bit of digging around I see that someone has created the Python based Mayan project (http://www.mayan-edms.com/). Part of me - the masochistic side wants to write a python based home brewed system, but if I only stick to using DOC-Pi for home paper work/Invoices etc. then really I am only looking at perhaps 100,000 to 200,000 documents. The Drupal CMS (https://www.drupal.org/), can easily handle this number of document and I know this package very well and it's fun to work with, so maybe I should try it out first.

At the moment though, my current thinking is that I am only going to use this to store photographed copies of invoices and document etc. These will be OCR'ed (Optical Character Recognition), into a text format and then the text will be partially processed via NLP (Natural Language Processing) Python package: NLTK (http://www.nltk.org/).

A few tips for people interested in this kind of thing. To OCR an image, you will need to install Tesseract (https://code.google.com/p/tesseract-ocr/). On a RPi you just need to run: sudo apt-get install tesseract-ocr

You can then OCR a jpeg file like this: tesseract 2015-11-14_182550.jpg 2015-11-14_182550

On a RPi 2, this can take 2-3 minutes per 500KB jpeg file (1900px by 2500px). The smaller the file, the faster it is. The more pixels per 'character' the better quality the OCR is, or less errors. It's good enough for what I need for the moment.

To install NLTK (Natural Language Tool Kit) software packages: sudo pip install nltk (if you don't have it, you need to install pip first: sudo apt-get install python-pip). Best then to download some supporting files:

python # run python from the RPi CLI
nltk_download() # pops up a menu, which I used to downloaded these files:
    averaged_perceptron_tagger
    punkt
    wordnet
    words
    toolbox
    brown

Here is a simple exploratory code to get you started:

Code: Select all

#
# this will import a text file and do some NLP (Natural Language Processing) on the words

# import all the python modules we need
import nltk
from nltk.corpus import wordnet

path = nltk.data.find('/home/pi/ocr/2015-11-14_182550.txt')

print "load file"
raw = open(path, 'rU').read()
print "decode"
words = raw.decode('unicode_escape').encode('ascii','ignore')
print "tokanize"
text = nltk.word_tokenize(words)
# analyse and tag the tokeized list of words
tag_text = nltk.pos_tag(text)
minlength = 3  # ignore words shorter than this
nouns = set()
pnouns = set()
verbs = set()
adjectives = set()
number = set()
symbol = set()
foreign = set()
url = set()

# this will create a list of various types of words and check that they are legitimate words
print "start processing:"
for word, pos in tag_text:
    if len(word) > minlength:
      # find web URLS
      if len(nltk.regexp_tokenize(word, r'(http://|https://|www.)[^"\' ]+')):
        url.add(word)
      if pos in ['SYM']:
        symbol.add(word)
      if pos in ['FW']:
        foreign.add(word)
      if pos in ['CD']:
        number.add(word)
      #print word
      if not wordnet.synsets(word):
        print word
      else:
        if pos in ['NN']:
          nouns.add(word)
        if pos in ['NNP']:
          pnouns.add(word)
        if pos in ['VBZ','VBG']:
          verbs.add(word)
        if pos in ['JJ']:
          adjectives.add(word)
print "Nouns: ", nouns
print "ProNouns: ", pnouns
print "Verbs: ", verbs
print "adjectives: ", adjectives
print "Numbers: ", number
print "Symbols: ", symbol
print "Foreign: ", foreign
print "URLs: ", url

# list out some frequency stats
print
print "Frequency Stats:"
fd = nltk.FreqDist(text)
print fd.keys()


After about 60 seconds of processing, the above script will display a lot of interesting info on the text file you fed into it.

At the moment, for my document 'keywords' I am going to go with the first 20 Nouns and Pronouns and any URLs that might be listed in the doc. It's very arbitrary and might change once I have a solid block of sample docs OCR'ed etc. Ideally, I would like it to extract pertinent info based on the kind of document, ie: a bank statement might only record date, bank name, account number, balance etc; Electricity statement might have date, company, amount, avg daily units used; an Insurance doc might record date, company, amount, policy number and policy update text etc. All of this info will be stored in a SQL DB, with a web interface.

Hopefully this will end up doing a bit more that an Evernote App.

User avatar
seaton
Master Maker
Posts: 222
Joined: Tue Jun 24, 2014 11:41 am
Location: Bunbury, WA
Contact:

Re: Doc-Pi

Post by seaton » Tue Nov 17, 2015 1:54 pm

Love it!

now you need a cluster to offload your OCRing :D

parkview
Guru Maker
Posts: 603
Joined: Tue Jun 24, 2014 8:25 pm
Location: Busselton
Contact:

Re: Doc-Pi

Post by parkview » Sun Nov 29, 2015 10:01 pm

Well, nothing like having a looming 2015 Robofair to motivate one to finish off the DOC-Pi electronics board.

I used KiCad to draw up a nicer circuit diagram than the hand drawn one above:
Doc-Pi Schematic V1.0a.jpg
Doc-Pi Schematic V1.0a.jpg (47.53 KiB) Viewed 17258 times


I realised that my trusty Traco Power TSR1-2450 voltage regulator wasn't going to power both the RPi and the RPi touch screen display, so I added an extra one just for the display.

So I have converted this ugly mess of wires:
Doc-Pi breadboard.jpg
Doc-Pi breadboard.jpg (226.68 KiB) Viewed 17258 times


Into this circuit board:
DOC-Pi Electronics board - Annotated.1.jpg
DOC-Pi Electronics board - Annotated.1.jpg (58.45 KiB) Viewed 17258 times


At the moment, this is just a once off prototype, so I am not going to bother building a circuit board for it, but I would be able to make it a lot smaller than the current hand wired board.

I have some power specs for the rig. I am currently running it off a 12V 7Ahr SLA battery:

  • 5.27W - running Doc-Pi.py with camera running at idle
  • 14.5W - running Doc-Pi.py and taking a photo with the LED flood lights on (only for a 1-2 seconds)

Using the above 2.2Kg battery to 50% capacity the rig should give me around 7 hours of run time.

Using 4 x 32650 Li-ion, 576gm batteries to 70% capacity the rig should give me around 9.5 hours of run time.

Post Reply