• Hail Guest!
    We're looking for Community Content Contribuitors to Stratics. If you would like to write articles, fan fiction, do guild or shard event recaps, it's simple. Find out how in this thread: Community Contributions
  • Greetings Guest, Having Login Issues? Check this thread!
  • Hail Guest!,
    Please take a moment to read this post reminding you all of the importance of Account Security.
  • Hail Guest!
    Please read the new announcement concerning the upcoming addition to Stratics. You can find the announcement Here!

NEWS NLS Newsletter #6

Pawain

I Hate Skilling
Governor
Stratics Veteran
Stratics Legend
UNLEASHED
VS is getting worse on all shards for some reason. I would bet it is processing power. There are bots doing searches on the internet to get websites to the top results. Google will drop you down if they discover you are doing that.
Funny how searches with proper processors can get results from millions of pages in milliseconds but UO can not get them from thousands.

I have been crafting all day and searching. I timed out many times on LS from 1 word searches.
I have also had another account searching on Atlantic while I craft. You can get things after many tries if you use a word in the search. That probably causes a circular problem since I am just searching over and over. x all the others searching.
To pass the time making max. I have the crafter doing searches on LS and another account searching on Atl.

If you are able to just use clickys the search is almost instant.

UO has a problem when you have to use an item name.
 

Kylie Kinslayer

Stratics Legend
Alumni
Stratics Veteran
Stratics Legend
UNLEASHED
Awards
1
True on the Google deal.. but they also have so many servers that it literally takes up acres of land. UO? Not so much.... Heck, I suspect their servers are all likely in a 10'x10' room :lol:
 

Lord Frodo

Stratics Legend
Stratics Veteran
Stratics Legend
UNLEASHED
It's just a poorly coded database, but if you still think your arguement regarding RMT TOOK DOWN VS CUS THINGS ARE THINGS AND BAD THINGS , then all the power to you

If the devs find an easy way to get out of the painfully obvious "we outsourced this job to someone who was not experienced in building a database of the necessary design", and it also appeases the equally easy to be swayed, then hang your hat on knowing you're an easy read.

This is text book stuff
LMMFAO You talk about our theory and the DEVs knowledge and what do you use as proof your own ******** theory as proof, what a ****ing joke.
 

Kylie Kinslayer

Stratics Legend
Alumni
Stratics Veteran
Stratics Legend
UNLEASHED
Awards
1
It's just a poorly coded database, but if you still think your arguement regarding RMT TOOK DOWN VS CUS THINGS ARE THINGS AND BAD THINGS , then all the power to you

If the devs find an easy way to get out of the painfully obvious "we outsourced this job to someone who was not experienced in building a database of the necessary design", and it also appeases the equally easy to be swayed, then hang your hat on knowing you're an easy read.

This is text book stuff
I will pose the same 2 questions to you that I posed to Pawain...

Can you prove it's not?

I mean the Devs seem to think that is the case.. and they have a lot more info on the subject than any of us ever will.. so, can you disprove what they say?
If that move fixes the issue will you admit it was correct?
I mean surely you have some kind of proof other than conjecture?? As for me, I don't claim to know either way as a fact... and I suspect not many players do.
 

petemage

Babbling Loonie
Stratics Veteran
Stratics Legend
UNLEASHED
It's just a poorly coded database, but if you still think your arguement regarding RMT TOOK DOWN VS CUS THINGS ARE THINGS AND BAD THINGS , then all the power to you

If the devs find an easy way to get out of the painfully obvious "we outsourced this job to someone who was not experienced in building a database of the necessary design", and it also appeases the equally easy to be swayed, then hang your hat on knowing you're an easy read.

This is text book stuff
This 100%. VS was a mess from day one (2014): Vendor Search Issues – Ultima Online

Ever since they kept applying bandaids over bandaids. Lipstick on a pig n such..
 

Coco_Zamis_DF

Lore Keeper
Governor
Stratics Veteran
Stratics Legend
UNLEASHED
Vs is about Bad coding and/or wrong technique. On a non Atlantic Server I bought an item yesterday and today it was still in the result list.

On the next server things I am seeking first Shows up when Changing the seek words. And I got a bigger list.
Even if two days before the other words works.
It could be that ej searching result in Performance issues but not in content.
 

Effort

Lore Keeper
Stratics Veteran
No @Kylie Kinslayer and @Lord Frodo that's not how accusations work.


You don't get to say "these specific people(RMT) are doing something to negatively impact the ENTIRE PLAYERBASE (ergo their potential customer base), and it's on you to prove they aren't!"


This is just dummies spitting on each other back and forth, blaming the space lasers for the rain.


I'm sure you're nice enough people but I'm checking out of this, best of luck
 

Spartan

Certifiable
Stratics Veteran
UNLEASHED
I will pose the same 2 questions to you that I posed to Pawain...





I mean surely you have some kind of proof other than conjecture?? As for me, I don't claim to know either way as a fact... and I suspect not many players do.
Actually, @Effort has some good points. I was an SQL database admin for 12 years and I can tell you this: any SQL database can have the tables and indices rebuilt all you want, but there is a command that (last I remember) is called UPDATE STATISTICS that also needs to be run.

See, the data tables in their indexes do fluctuate a lot, and when - as in indexes - dropped and then re-added to rebuild itself, there are system-controlled areas that also need to be updated .... or things go to $hit in a hurry. That's what the update-stats command does.

If this is NOT done, slow response, incorrect response or even failure to respond can happen. Admittedly, the databases I controlled were business app stuff, but the process and needs are the same within the same product. This is all predicated on SQL being the database engine of choice here - AND foreign keys, internal procs, constraints, etc are properly setup..
 
Last edited:

Theo_GL

Grand Poobah
Stratics Veteran
Stratics Legend
UNLEASHED
Actually, @Effort has some good points. I was an SQL database admin for 12 years and I can tell you this: any SQL database can have the tables and indices rebuilt all you want, but there is a command that (last I remember) is called UPDATE STATISTICS that also needs to be run.

See, the data tables in their indexes do fluctuate a lot, and when - as in indexes - dropped and then re-added to rebuild itself, there are system-controlled areas that also need to be updated .... or things go to $hit in a hurry. That's what the update-stats command does.

If this is NOT done, slow response, incorrect response or even failure to respond can happen. Admittedly, the databases I controlled were business app stuff, but the process and needs are the same within the same product. This is all predicated on SQL being the database engine of choice here - AND foreign keys, internal procs, constraints, etc are properly setup..
Updating the statistics forces the DB to recount the number of records in the tables. This is important for efficiency when joining tables because the statistics are used by the cost based optimizer to determine when to use an index or just table scan.
I’d have to see how VS was implemented but they do have a challenge in that item properties don’t fit into a nice pattern. Weapons have different properties from deco compared to scrolls etc. Based on the sort/filter options they are stripping out common properties into an indexed property table. That’s why when you use the menus to search for ‘LRC 2.0’ it finds it quickly by hitting the index on the property table.
The problem is they allow a full text search and it’s on every part of the item instead of just a description field. So when you type in ‘alacrity taming’ it’s scanning every piece of text in every item right down to the ,durability 255/255. That’s crazy expensive on a database. They need to do three things to fix it :
1- Add filters for Power Scroll, Alacrity and Trans scrolls cutting down on a lot of VS text searches. The fact this was not a primary filter is a complete dev failure.
2- Restrict text search to only the item description - not every part of the item. Ie if you search for Lower Reagent you won’t get any items because it’s not in the description it’s in the properties. Filter on that property instead. This will hugely reduce the sheer amount of text it has to match on for a text search.
3- Last always force someone to select between (armor weapons jewels), (Scrolls and deeded items) and (misc items). This would always segment the databases by a third.Most of the time when you search you know the type of object you are after. By allowing the DB to cut down the records by2/3 would increase efficiency by 67%. Immediately.
But hey what do I know. Just managed the worlds data for 20+ years. I offered to help. For free.
 

petemage

Babbling Loonie
Stratics Veteran
Stratics Legend
UNLEASHED
Updating the statistics forces the DB to recount the number of records in the tables. This is important for efficiency when joining tables [...]
Actually, @Effort has some good points. I was an SQL database admin for 12 years [...]
Just what makes you think this stuff is even in a relational database?

I rather guess they build some homebrew system on top of their thousands plain world object files. SearchUO and all those sites also indexed all Luna and beyond and never had any issues with querying stuff. IMO the poor performance of VS can't be explained by having forgot to set some indices on a bunch of table columns but is way worse. Even Kyronix/Bleak would have figured to add those by now, but it seems it's not that easy.
 
Last edited:

Dan(Tasanar)

Journeyman
Professional
UNLEASHED
On emulation Vendor Search is simply coded in C# to run in its own thread.

using System.Threading.Tasks;
There is no database in the sense of what you guys are talking about here.

When testing and coding it we simulated about a million items on vendors with a few hundred requests for searching going off every second.
(We just coded a test script on the side to populate all PlayerVendors in the world with randomly priced trash and then flooded it with searches)

Response time was within a second or two

We also streamlined string searching / basic text input.
Code:
public static string GetItemName(Item item)
        {
            if (StringList == null || item.Name != null)
                return item.Name;

            ObjectPropertyList opl = new ObjectPropertyList(item);
            item.GetProperties(opl);

            //since the object property list is based on a packet object, the property info is packed away in a packet format
            byte[] data = opl.UnderlyingStream.UnderlyingStream.ToArray();

            int index = 15; // First localization number index
            string basestring = null;

            //reset the number property
            uint number = 0;

            //if there's not enough room for another record, quit
            if (index + 4 >= data.Length)
            {
                return null;
            }

            //read number property from the packet data
            number = (uint)(data[index++] << 24 | data[index++] << 16 | data[index++] << 8 | data[index++]);

            //reset the length property
            ushort length = 0;

            //if there's not enough room for another record, quit
            if (index + 2 > data.Length)
            {
                return null;
            }

            //read length property from the packet data
            length = (ushort)(data[index++] << 8 | data[index++]);

            //determine the location of the end of the string
            int end = index + length;

            //truncate if necessary
            if (end >= data.Length)
            {
                end = data.Length - 1;
            }

            //read the string into a StringBuilder object

            StringBuilder s = new StringBuilder();
            while (index + 2 <= end + 1)
            {
                short next = (short)(data[index++] | data[index++] << 8);

                if (next == 0)
                    break;

                s.Append(Encoding.Unicode.GetString(BitConverter.GetBytes(next)));
            }

            basestring = StringList.GetString((int)number);
            string args = s.ToString();

            if (args == string.Empty)
            {
                return basestring;
            }

            string[] parms = args.Split('\t');

            try
            {
                if (parms.Length > 1)
                {
                    for (int i = 0; i < parms.Length; i++)
                    {
                        parms[i] = parms[i].Trim(' ');

                        if (parms[i].IndexOf("#") == 0)
                        {
                            parms[i] = StringList.GetString(Convert.ToInt32(parms[i].Substring(1, parms[i].Length - 1)));
                        }
                    }
                }
                else if (parms.Length == 1 && parms[0].IndexOf("#") == 0)
                {
                    parms[0] = StringList.GetString(Convert.ToInt32(args.Substring(1, parms[0].Length - 1)));
                }
            }
            catch
            {
                return null;
            }

            StringEntry entry = StringList.GetEntry((int)number);

            if (entry != null)
            {
                return entry.Format(parms);
            }

            return basestring;
        }

Broadsword has specifically used the word database regarding VS a bit so obviously totally different, just showing another way. (I'd post the link directly to the emulation vendor search code but I'm not sure that's allowed here.)
 
Last edited:

Theo_GL

Grand Poobah
Stratics Veteran
Stratics Legend
UNLEASHED
Just what makes you think this stuff is even in a relational database?

I rather guess they build some homebrew system on top of their thousands plain world object files. SearchUO and all those sites also indexed all Luna and beyond and never had any issues with querying stuff. IMO the poor performance of VS can't be explained by having forgot to set some indices on a bunch of table columns but is way worse. Even Kyronix/Bleak would have figured to add those by now, but it seems it's not that easy.
Given they have said it was down for Database maintenance was my clue.
However it is not the only way to solve the issue just the most traditional.
Also regarding someone mentioning how old and archaic UO severs must be that’s not true as they migrated to AWS years ago. If the DB is there too that gives them unlimited scalability of hardware so further embarrassing that performance is spotty.
 

petemage

Babbling Loonie
Stratics Veteran
Stratics Legend
UNLEASHED
Given they have said it was down for Database maintenance was my clue.
However it is not the only way to solve the issue just the most traditional.
Also regarding someone mentioning how old and archaic UO severs must be that’s not true as they migrated to AWS years ago. If the DB is there too that gives them unlimited scalability of hardware so further embarrassing that performance is spotty.
I've seen people call MS Access and even their extensive Excel sheets a "database" :D Who knows but I doubt the bad performance literally since day one comes just from a badly configured RDBMS.
 

petemage

Babbling Loonie
Stratics Veteran
Stratics Legend
UNLEASHED
Did they just unlist the upcoming youtube stream? You can only reach it through the link in the newsletter now, but not through the youtube channel itself. It was public yesterday. Bonnie already hurt before it even started :lol:

I gotta leave the link here to save me a click later..

 

Uriah Heep

Grand Poobah
Alumni
Stratics Veteran
Stratics Legend
UNLEASHED
Did they just unlist the upcoming youtube stream? You can only reach it through the link in the newsletter now, but not through the youtube channel itself. It was public yesterday. Bonnie already hurt before it even started :lol:

I gotta leave the link here to save me a click later..

As far as I can see, they hyavent invested any time in advertising it other than that one brief mention..
 
Top