Travel the dunes with the LEGO� Star Wars™ Ultimate Collector Series Sancrawler™

FBTB - From Bricks To Bothans

Follow us: RSS
News? Questions? Comments? Email!

Help with Bricklink

Want to talk about how the LEGO company is doing, your recent visit to LEGOland, your local LEGO club, other resources in the online LEGO community, or about LEGO software and games? Come in here and join us!

Help with Bricklink

Postby Nick2253 » Fri Jan 13, 2012 8:23 pm

I'm not even sure where to ask this, but I figured FBTB is as good a place as any:

Having used Bricklink for a number of years, I can honestly say that I hate the interface. Unfortunately, there are no serious competitors in town, so we get what we get. However, Bricklink does have all the data that I would need in order to determine everything I want.

So, the question: does anyone know if BrickLink provides an API in order for third-party programs to access and download their database of parts and seller information, including price and quantity of parts for sale? I know you can manually download the catalog, which would be fine if that's all I needed, but it's not. I would also like to be able to download/upload wishlists to the site.

My final goal is to code (or find--could I get so lucky as to have someone who already did this?) a program that allows me to specify a set of parts, and then, based on the properties of those parts and the different BrickLink stores, the program automatically determines the optimal store or stores to buy from in order to minimize the total cost (including at least a rough shipping estimate). Algorithmically, I know exactly how to do all of this; I just need the data.

Right now, I'm stuck trawling through the stores manually, and make a greedy guess as to which stores will give me the lowest cost.

If you have any advice, pointers, or recommendations, I'm all ears!

Thanks,
Nick
Nick2253
 
Posts: 14
Joined: Thu Jan 05, 2012 3:44 pm
Location: SF Bay Area, CA

Re: Help with Bricklink

Postby AbstractedIdeal » Fri Jan 13, 2012 9:03 pm

I don't know if there's an official API, but you might want to look at how BrickStore pulls info. I'm not sure if it pulls Seller info, but you might be able to co-opt or adapt whatever methods it's using.

Nevermind -- looks like BrickStore pulls its data from the author's own site, so you're likely stuck with screen scraping the data.
AbstractedIdeal
 
Posts: 76
Joined: Fri Dec 04, 2009 12:07 am

Re: Help with Bricklink

Postby GIR3691 » Sat Jan 14, 2012 1:49 am

Have you tried making a wish list of the parts you want? It's not ideal, but you can select all categories, and then "show all." It will give you a list of stores that stock each item on your wish lsit. Refine your parameters and you can look for store names that pop up for multiple things.

In my experience, you'll never find everything you want in one store. I usually end up making two or three orders whenever working on a specific project. Try to go by the rarest/ highest priority items first, then see what stores stock them. From there you can browse their inventory to see if they have other parts you want, and you can usually narrow it down to a couple of stores to get the most out of each order.

Hope this helps somewhat. I too have thought about how nice it would be to make a wishlist and have it generate recommended stores automatically.
Image
>>[email protected] <<
GIR3691
Staff Writer
 
Posts: 715
Joined: Tue Feb 10, 2009 8:33 pm
Location: CT

Re: Help with Bricklink

Postby bigospedros » Sat Jan 14, 2012 4:46 am

Wouldn't it be a good idea to approach the owners of Bricklink and talk to them about your proposed idea. They may work with you to incorporate it into the site ?
bigospedros
Staff Writer
 
Posts: 650
Joined: Wed Feb 11, 2009 4:45 am
Location: Waterlooville, UK

Re: Help with Bricklink

Postby Nick2253 » Mon Jan 16, 2012 11:06 pm

I currently use their wishlist in order to find what I need, and there are actually more efficient ways to determine what I want on BrickLink than your method, but it's still incredibly tedious. I always seem to be designing models that use some of the rarer parts, so I know what you mean about placing multiple orders. However, I have to do lots of legwork in order to determine the best store for my orders, and when all is said and done, I'm putting in a lot of work for minimal savings. Yes, it does add up, so it is worth it, but it really is a pain.

I actually never thought to approach BrickLink directly. I'll have to do that.

Thanks all!
Nick2253
 
Posts: 14
Joined: Thu Jan 05, 2012 3:44 pm
Location: SF Bay Area, CA

Re: Help with Bricklink

Postby legodavee123 » Tue Jan 17, 2012 7:00 pm

bigospedros wrote:Wouldn't it be a good idea to approach the owners of Bricklink and talk to them about your proposed idea. They may work with you to incorporate it into the site ?


Pretty difficult. Since Dan passed on, his family has inherited the website, and I'm not sure anyone that's still there is able to do any difficult code work. I'm not sure if there's anyone doing any coding at BL at the moment-- it's possible, but I'm not aware of anyone doing that...

DaveE
legodavee123
 
Posts: 82
Joined: Thu Jul 16, 2009 11:50 am

Re: Help with Bricklink

Postby RfAurora » Wed Jan 18, 2012 12:35 pm

legodavee123 wrote:
bigospedros wrote:Wouldn't it be a good idea to approach the owners of Bricklink and talk to them about your proposed idea. They may work with you to incorporate it into the site ?


Pretty difficult. Since Dan passed on, his family has inherited the website, and I'm not sure anyone that's still there is able to do any difficult code work. I'm not sure if there's anyone doing any coding at BL at the moment-- it's possible, but I'm not aware of anyone doing that...

DaveE

It's not difficult at all. If you haven't yet, read the notification under the login/userid box in the top right corner of Bricklink. Not only is it a memorial to Dan, but it also gives information on who is maintaining the website, and down at the bottom of the page is a link to Northstar Computer Systems where the new moderator works.
RfAurora
 
Posts: 149
Joined: Thu Sep 17, 2009 7:17 pm

Re: Help with Bricklink

Postby Nick2253 » Wed Jan 18, 2012 1:24 pm

I got back an official reply from Bricklink:

Bricklink wrote:We don't have anything available at this time. We are considering doing this for the future, but it is not an immediate project.


I guess that answers that. My next option is to create a web-trawling script that will index all the store for me. It will take time, and unfortunately won't be able to update super rapidly, but should be good enough for what I need.
Nick2253
 
Posts: 14
Joined: Thu Jan 05, 2012 3:44 pm
Location: SF Bay Area, CA

Re: Help with Bricklink

Postby legodavee123 » Fri Jan 20, 2012 9:34 am

RfAurora wrote:It's not difficult at all. If you haven't yet, read the notification under the login/userid box in the top right corner of Bricklink. Not only is it a memorial to Dan, but it also gives information on who is maintaining the website, and down at the bottom of the page is a link to Northstar Computer Systems where the new moderator works.


Maintaining != Coding.

Basically, yes, there are a lot of people who are maintaining the site. They're using the interfaces that Dan wrote to keep the database current, and deal with user problems using those same interfaces. Same way it's been going for years. Similarly, if there are any network or operating system issues, they've got staff at NorthStar to deal with it.

But from what I've heard, new development is difficult, correct me if I'm wrong. If, say, they wanted to implement some new feature to the website, that means digging into the code that Dan wrote, and (last I heard) there wasn't anyone who was familiar enough with the code to do that.

DaveE
legodavee123
 
Posts: 82
Joined: Thu Jul 16, 2009 11:50 am

Re: Help with Bricklink

Postby TheCreativeBrickCoUk » Fri Feb 03, 2012 6:41 am

Hi,

I have written a program which does what you require although it doesn't take into account shipping costs - which can make quite a difference to where the the best prices come from. It basically a web-scraping application - I don't run it
very often - perhaps once a month - and I don't require it to cover all of the BL site (although it could).

The following text will give you an idea of what is does for a particular Lego set inventory.
(In this case limited to UK suppliers)

[SAMPLEOUTPUTSTART]

Part 3455, Brick, Arch 1 x 6 (Black) - 4 Required
Found All (4) at Bricks For U UK - B7 (UK) : $0.03 (Used)
Running Totals : 0.92

Part 3023, Plate 1 x 2 (Light Bluish Gray) - 8 Required
Found Some (4) at BricksNParts - BP (UK) : $0.02 (Used)
Found Remainder (4) at Brick Babe - BB (UK) : $0.03 (Used)
Running Totals : 9.10

Part 3623, Plate 1 x 3 (Light Bluish Gray) - 4 Required
Found All (4) at A Mix Of Bricks - MK (UK) : $0.03 (Used)
Running Totals : 9.22

Part 3666, Plate 1 x 6 (Light Bluish Gray) - 15 Required
Found Some (1) at Bricks For U UK - B7 (UK) : $0.04 (Used)
Found Some (10) at Bricks_4_Sale - B9 (UK) : $0.05 (Used)
Found Remainder (4) at Brick Babe - BB (UK) : $0.05 (Used)
Running Totals : 9.99

Summary:

Short By 153 Pieces
Total Pieces : 902
Total Price : $48.08501 (£26.86313)

[SAMPLEOUTPUTEND]



I'm afraid the code is fine for my needs but isn't in any sort of state which I would be happy about releasing.

As a website owner myself I know what a pain scraping programs can be so as I've indicated I only run it every few weeks and it is very 'gentle' - pausing for 10 seconds between each page retrieval - which makes it slow - but kind to BL.

What would be great would be a datafeed or datadump from BL - although they may place too high a value on their data to consider this...

Rob

TheCreativeBrickCoUk

http://www.thecreativebrick.co.uk
TheCreativeBrickCoUk
 
Posts: 2
Joined: Thu Feb 02, 2012 12:22 pm


Return to General LEGO Discussions

Who is online

Users browsing this forum: Exabot [Bot] and 4 guests