Query caching in controller - ruby-on-rails

Rails 3.2.18
ruby 1.9.3
redis
checking caching in development environment.
development.rb:
S2yd::Application.configure do
# Settings specified here will take precedence over those in config/application.rb
# In the development environment your application's code is reloaded on
# every request. This slows down response time but is perfect for development
# since you don't have to restart the webserver when you make code changes.
config.cache_classes = false
#config.assets.enabled = true
# Log error messages when you accidentally call methods on nil.
config.whiny_nils = true
# Show full error reports and disable caching
config.consider_all_requests_local = true
config.action_controller.perform_caching = false
# Don't care if the mailer can't send
config.action_mailer.raise_delivery_errors = false
# Print deprecation notices to the Rails logger
config.active_support.deprecation = :log
# Only use best-standards-support built into browsers
config.action_dispatch.best_standards_support = :builtin
config.cache_store = :redis_store, 'redis://localhost:6379/0/cache', { expires_in: 90.minutes, namespace: 'production' }
controller
#restaurants = Rails.cache.fetch("api/restaurants/#{#thirdparty.name}/1", expires_in: 90.minutes) do
Restaurant
.joins(:thirdparties, :franchise)
.includes(:menu_sections, :menus)
.where("`thirdparties`.`id` = ? and `restaurants`.`active` = ? and lower(`restaurants`.`name`) <> ?", #thirdparty.id, true, 'drinks')
end
log
Processing by Api::V1Controller#restaurants as JSON
Parameters: {"token"=>"eatstreet"}
(0.5ms) SELECT COUNT(*) FROM `domains`
Domain Load (0.6ms) SELECT `domains`.* FROM `domains`
Domain Load (0.5ms) SELECT `domains`.* FROM `domains` LIMIT 1
Domain: 127.0.0.1 -> S2YD
User Load (0.6ms) SELECT `users`.* FROM `users` WHERE `users`.`id` = 93626 LIMIT 1
Thirdparty Load (0.5ms) SELECT `thirdparties`.* FROM `thirdparties` WHERE `thirdparties`.`access_token` = 'eatstreet' LIMIT 1
Thirdparty: eatstreet
Restaurant Load (21.7ms) SELECT `restaurants`.* FROM `restaurants` INNER JOIN `restaurants_thirdparties` ON `restaurants_thirdparties`.`restaurant_id` = `restaurants`.`id` INNER JOIN `thirdparties` ON `thirdparties`.`id` = `restaurants_thirdparties`.`thirdparty_id` INNER JOIN `franchises` ON `franchises`.`id` = `restaurants`.`franchise_id` WHERE (`thirdparties`.`id` = 3 and `restaurants`.`active` = 1 and lower(`restaurants`.`name`) <> 'drinks')
Menu Load (10.2ms) SELECT `menus`.* FROM `menus` WHERE `menus`.`restaurant_id` IN (4, 6, 7, 8, 10, 12, 14, 286, 287, 288, 322, 352, 353, 355, 394, 467, 480, 503, 523, 552, 553, 555, 615, 641, 650, 744, 785, 790, 793, 797, 799, 800, 801, 802, 807, 834, 835, 863, 865, 882, 954, 1012, 1014, 1110, 1111, 1114, 1128, 1139, 1188, 1191, 1196, 1198, 1202, 1206, 1225, 1297, 1325, 1358, 1379, 1387, 1404, 1436, 1509, 1510, 1519, 18, 39, 46, 49, 62, 66, 67, 86, 112, 129, 199, 203, 205, 209, 228, 296, 305, 306, 365, 368, 381, 382, 384, 390, 395, 496, 501, 510, 561, 565, 584, 607, 694, 767, 825, 839, 857, 926, 946, 947, 970, 996, 1000, 1028, 1040, 1055, 1071, 1094, 1109, 1160, 1216, 1244, 1339, 1355, 1356, 1364, 1375, 1418, 1438, 1448, 1508, 1518, 1527, 1564, 131, 132, 134, 135, 139, 142, 218, 249, 308, 309, 310, 324, 339, 350, 351, 361, 363, 367, 369, 370, 392, 396, 400, 490, 581, 585, 597, 836, 849, 993, 1118, 1119, 1122, 1140, 1173, 1218, 1226, 1245, 1246, 1247, 1248, 1249, 1254, 1255, 1259, 1260, 1262, 1266, 1272, 1301, 1308, 1316, 1318, 1335, 1351, 1405, 1408, 1451, 1513, 146, 149, 152, 155, 160, 161, 163, 165, 167, 190, 193, 194, 283, 291, 294, 334, 345, 346, 347, 360, 388, 426, 427, 519, 536, 577, 588, 606, 712, 798, 921, 1043, 1155, 1238, 1258, 1287, 1376, 1381, 1385, 1386, 1413, 1417, 1430, 1433, 1440, 1441, 1479, 1480, 1481, 1482, 1558, 255, 256, 259, 276, 446, 453, 456, 458, 460, 461, 462, 464, 465, 466, 468, 470, 471, 474, 491, 494, 514, 558, 626, 683, 697, 730, 748, 780, 786, 842, 858, 1015, 1161, 1197, 1357, 1371, 1373, 1444, 1453, 374, 377, 378, 401, 498, 499, 528, 564, 573, 636, 809, 909, 933, 944, 407, 409, 417, 455, 476, 479, 517, 560, 591, 604, 709, 754, 771, 833, 1072, 1073, 1074, 1099, 1185, 419, 420, 436, 437, 440, 441, 492, 507, 508, 605, 634, 648, 651, 655, 710, 729, 824, 860, 1181, 1207, 1227, 1239, 1253, 1269, 1270, 1271, 1286, 1310, 1311, 1312, 1313, 1321, 1322, 1343, 1344, 1345, 1346, 1382, 1383, 1416, 488, 543, 568, 571, 598, 622, 625, 724, 1022, 1235, 1288, 1298, 1299, 1302, 1320, 1323, 1334, 1349, 1412, 1429, 1439, 1449, 1450, 1454, 1486, 1515, 1516, 1517, 1520, 1523, 529, 537, 546, 590, 608, 609, 610, 624, 680, 681, 685, 716, 831, 1150, 1151, 1153, 1178, 1186, 1363, 1512, 1514, 638, 695, 715, 717, 718, 719, 753, 781, 837, 845, 848, 850, 960, 1027, 1129, 1233, 1250, 1303, 1367, 1423, 1434, 1442, 1468, 629, 631, 656, 658, 659, 665, 669, 741, 760, 769, 778, 908, 910, 915, 916, 917, 927, 937, 938, 941, 942, 943, 955, 957, 959, 972, 973, 974, 975, 976, 977, 979, 980, 992, 994, 999, 1005, 1023, 1134, 1157, 1174, 1208, 1210, 1224, 1229, 1242, 1304, 1314, 1315, 1378, 1415, 1424, 1435, 1445, 1492, 675, 1041, 1042, 1045, 1046, 1060, 1061, 1064, 1067, 1069, 1080, 1081, 1091, 1101, 1102, 1103, 1123, 1156, 1166, 1222, 1230, 1353, 1365, 1366, 1403, 1410, 1414, 1431, 1443, 1500, 1521, 722, 761, 764, 766, 768, 770, 805, 821, 822, 859, 928, 929, 930, 931, 932, 1059, 1106, 1115, 1234, 1352, 1447, 728, 731, 732, 734, 737, 738, 750, 777, 782, 794, 795, 818, 829, 841, 1029, 1063, 1068, 1105, 1182, 1183, 1256, 1275, 1278, 1348, 1372, 1406, 1566, 854, 861, 862, 867, 874, 880, 881, 890, 893, 894, 895, 905, 936, 956, 1006, 1011, 1013, 1024, 1026, 1030, 1033, 1038, 1044, 1116, 1117, 1121, 1165, 1192, 1193, 1209, 1309, 876, 877, 878, 879, 899, 902, 907, 913, 1004, 1050, 1078, 1079, 1086, 1107, 1108, 1199, 1243, 1292, 1411, 1457, 1458, 1459, 1460, 1461, 1465, 1466, 1467, 1470, 1472, 1474, 1477, 1526, 1538, 1568, 1569, 1570, 1572, 1573, 1574, 1576, 1577, 1578, 1582, 1583, 1584)
MenuSection Load (44.5ms) SELECT `menu_sections`.* FROM `menu_sections` WHERE `menu_sections`.`menu_id` IN (4, 160, 1885, 7, 8, 121, 10, 319, 12, 14, 130, 358, 2225, 2226, 251, 253, 305, 367, 368, 374, 447, 536, 2242, 554, 591, 618, 652, 663, 1812, 1826, 2235, 2257, 745, 1857, 804, 896, 971, 1798, 1000, 2455, 1816, 2418, 1015, 1005, 1010, 1009, 2389, 1019, 1082, 1072, 1903, 1099, 1104, 1113, 2477, 1246, 1346, 1357, 1518, 1519, 2390, 1525, 1539, 2196, 1555, 1643, 1659, 1664, 1668, 1917, 2557, 2559, 1843, 1900, 2292, 2088, 2112, 2116, 2173, 2198, 2180, 2191, 2272, 2406, 2412, 2490, 2408, 2444, 2445, 2446, 2447, 2448, 2449, 171, 172, 175, 2574, 21, 1092, 1889, 1976, 2011, 22, 669, 24, 25, 26, 686, 27, 2558, 351, 352, 31, 1018, 33, 129, 242, 1735, 1762, 1763, 95, 98, 1787, 1792, 1871, 2159, 2161, 2162, 2496, 2497, 103, 478, 1027, 148, 256, 1733, 1770, 2245, 1723, 1883, 505, 2526, 392, 399, 400, 1064, 1066, 1067, 414, 415, 416, 435, 1197, 1200, 2139, 1811, 1872, 1873, 1876, 1878, 1898, 587, 588, 590, 1326, 1979, 1987, 612, 615, 672, 680, 1841, 735, 1925, 865, 2499, 2500, 2502, 2503, 2505, 1058, 1942, 1943, 1190, 1191, 1606, 2293, 1097, 1217, 1241, 1349, 1273, 1275, 1276, 1277, 1278, 1279, 1327, 1649, 1650, 1651, 1652, 1653, 2106, 2107, 2280, 2281, 2309, 2310, 1387, 1395, 1425, 1426, 1436, 1457, 1495, 1523, 1617, 1619, 1793, 1794, 1780, 1781, 1782, 1888, 1918, 2097, 2127, 2108, 2115, 2125, 2128, 2163, 2437, 2239, 2284, 2338, 2433, 2393, 2399, 2410, 2421, 2462, 2584, 35, 265, 1116, 2054, 2113, 1696, 1751, 43, 46, 132, 2114, 2211, 2212, 2214, 2436, 2440, 287, 671, 673, 1542, 1703, 292, 2080, 309, 344, 346, 364, 2228, 366, 381, 384, 2146, 391, 1802, 393, 394, 1710, 443, 448, 2419, 568, 1186, 711, 2602, 2603, 702, 1372, 1075, 1129, 1757, 1321, 1529, 1766, 1884, 1567, 1724, 1725, 2650, 2651, 1796, 1797, 1836, 1837, 1926, 1928, 1931, 1929, 1932, 1934, 2342, 2343, 1980, 1946, 2523, 2010, 1950, 1951, 2219, 2060, 2018, 2035, 2034, 2079, 2575, 2164, 2197, 2531, 2203, 2344, 2405, 50, 395, 2150, 52, 1879, 2008, 55, 58, 62, 63, 65, 67, 159, 167, 398, 1702, 86, 89, 90, 238, 239, 240, 247, 252, 337, 2474, 1803, 2613, 545, 550, 1823, 1824, 1587, 1902, 2611, 480, 481, 630, 637, 2493, 2264, 2612, 701, 734, 1867, 870, 1448, 1325, 1455, 2000, 2279, 1579, 1863, 2546, 1936, 1940, 1974, 2223, 2176, 2190, 2555, 2643, 2179, 2216, 2222, 2178, 2215, 2217, 2675, 2232, 2256, 2261, 2295, 2582, 2321, 2352, 2353, 2396, 2397, 2364, 2371, 2372, 2560, 1127, 198, 2271, 2274, 230, 499, 1984, 1985, 519, 2204, 1989, 527, 528, 529, 1765, 533, 534, 539, 541, 542, 548, 569, 570, 2265, 2266, 2004, 2006, 1817, 766, 854, 863, 864, 894, 1905, 2529, 962, 978, 1079, 1916, 2556, 1359, 1604, 2282, 1821, 2109, 2111, 2154, 2155, 2480, 2313, 2341, 2348, 423, 425, 2233, 2247, 421, 2134, 2135, 2136, 420, 450, 452, 581, 2335, 583, 584, 678, 799, 1400, 683, 687, 708, 1418, 771, 822, 1033, 1175, 2165, 1239, 1259, 1265, 458, 2573, 460, 472, 518, 565, 551, 1865, 2229, 553, 613, 676, 715, 2206, 2498, 856, 2409, 872, 918, 953, 954, 958, 959, 960, 963, 2562, 2563, 2564, 2565, 2566, 1070, 1458, 1459, 1755, 1501, 1736, 1636, 470, 1911, 490, 1071, 2607, 1963, 1964, 494, 506, 1324, 2463, 2600, 2601, 600, 1743, 751, 1679, 1691, 800, 1795, 2147, 2646, 814, 1699, 817, 818, 868, 1632, 1052, 1103, 1778, 1640, 1708, 1838, 1868, 1870, 1933, 1956, 1960, 2609, 1959, 2608, 2007, 2019, 2028, 2645, 2020, 2610, 2021, 2022, 2059, 2057, 2102, 2103, 2089, 2090, 2181, 2182, 2236, 1734, 2071, 2101, 2083, 1832, 1464, 1808, 1927, 724, 760, 770, 782, 885, 2407, 1969, 1970, 1971, 1861, 2156, 2031, 2030, 2032, 2033, 2046, 2055, 2056, 2048, 2063, 2314, 2218, 2385, 2294, 2328, 2339, 2350, 2382, 2383, 2359, 2411, 2414, 2415, 2413, 2452, 2416, 2438, 2430, 632, 653, 2401, 655, 679, 2244, 2604, 2539, 2540, 750, 1534, 1839, 1840, 769, 883, 2606, 943, 2605, 866, 874, 1068, 1086, 1576, 1575, 2395, 1578, 1625, 1638, 1827, 2124, 2402, 2441, 2521, 907, 903, 873, 2188, 875, 2289, 880, 1470, 2423, 2442, 891, 2479, 2519, 916, 966, 2095, 2595, 1083, 1968, 1685, 2243, 1088, 2476, 1263, 1272, 1383, 2495, 1541, 1784, 1999, 2596, 1935, 2494, 1996, 2543, 2213, 2520, 2250, 2262, 2434, 2327, 2422, 2384, 798, 2062, 2210, 2532, 831, 823, 908, 829, 1845, 834, 1858, 895, 1666, 932, 933, 945, 946, 969, 1767, 1901, 1586, 1834, 2224, 1180, 1288, 1163, 1164, 1166, 1165, 1201, 1223, 1222, 1225, 1227, 1228, 1371, 1247, 1690, 1552, 1257, 1282, 1375, 1378, 2142, 2143, 1304, 2398, 1290, 1362, 1629, 1631, 1941, 1785, 1789, 1801, 2185, 1286, 1295, 1420, 1421, 1292, 1294, 2337, 1340, 1726, 2283, 1374, 1558, 1623, 1682, 1639, 1709, 1768, 1769, 1831, 1848, 1904, 1998, 2220, 2027, 2029, 2172, 2199, 2227, 2249, 2269, 2325, 2332, 2367, 1403, 1396, 1507, 1509, 1510, 1397, 1566, 2622, 2623, 1404, 1405, 1428, 2168, 1438, 2486, 1443, 2567, 2590, 1453, 1472, 1481, 2588, 2589, 1475, 1496, 1503, 2585, 1505, 2591, 1506, 1532, 1591, 2315, 2316, 2317, 2318, 2319, 2320, 1605, 1819, 1846, 2096, 2133, 2131, 2192, 2202, 2221, 2268, 2312, 2365, 2400, 2417, 2427, 2428, 884, 935, 941, 1687, 1850, 944, 2465, 947, 1017, 1353, 2470, 1049, 1047, 1048, 1105, 1777, 1237, 2580, 2581, 1236, 1230, 1234, 2597, 2598, 1231, 1232, 2578, 1882, 1521, 1783, 1528, 1860, 2149, 2336, 920, 2471, 1669, 2002, 2267, 2647, 928, 1497, 2511, 2512, 2513, 2514, 2515, 997, 936, 919, 956, 1323, 1986, 2148, 2472, 1694, 2484, 996, 998, 1040, 1063, 1078, 1386, 1389, 1437, 1444, 2370, 1513, 1514, 1633, 1634, 1938, 1952, 1966, 2545, 2158, 2193, 2644, 1145, 1188, 2458, 1154, 1572, 1158, 1159, 1658, 2259, 2530, 1148, 1176, 1150, 1160, 1169, 2340, 1173, 1181, 1194, 1195, 1172, 1206, 2522, 1177, 1179, 1252, 2119, 1262, 1351, 2349, 2594, 1360, 1367, 1385, 1381, 2288, 2291, 2329, 2330, 1410, 1467, 1540, 1527, 1551, 1624, 2461, 1663, 1660, 2049, 2052, 2024, 2025, 1107, 1149, 1132, 1168, 1114, 1135, 1143, 1967, 1973, 1174, 1184, 1334, 1434, 1465, 1531, 1490, 1491, 2047, 1516, 1515, 1522, 1707, 1912, 2085, 2205, 2376, 2379, 2380, 2387, 2391, 2386, 2392, 2355, 2394, 2425, 2381, 2354, 2360, 2361, 2374, 2454, 2488, 2616, 2627, 2619, 2624, 2625, 2639, 2626, 2629, 2630, 2631, 2634, 2635, 2636) ORDER BY `index` ASC
Completed 200 OK in 3321.9ms (Views: 3235.1ms | ActiveRecord: 79.0ms)
Seems Rails.cache.fetch doesn't work and still touches the database.

I think you should change the config.
config.action_controller.perform_caching = false
Try to change false to true.

Related

How to identify Bimodal Histograms automatically?

I have a problem where I receive a lot of histograms from some database images. Those histograms are represented as vectors (0...255), and I have to identify and work with the bimodal histograms.
Is there a formula to automatically identify which histograms are bimodal and which aren't? Since they are numeric vectors, I could use a programming language (Java/C#) to work with it.
Is there a criterion on literature to identify bimodal histograms by software?
Here are 3 examples of histograms and format inputs I'm working with. Each histogram is a vector with 256 (0...255) positions.
Histogram 1
8029, 41, 82, 177, 135, 255, 315, 591, 949, 456, 499, 688, 446, 733, 712, 1595, 2633, 3945, 6134, 9755, 9236, 11911, 11888, 9450, 13119, 8819, 5991, 4399, 6745, 2017, 3747, 1777, 2946, 1623, 2151, 454, 3015, 3176, 2211, 1080, 391, 580, 750, 473, 10424, 334, 559, 621, 340, 2794, 1094, 5274, 2822, 204, 389, 728, 268, 15, 1060, 58, 113, 2728, 52, 3166, 11, 103, 522, 107, 351, 97, 66, 565, 315, 444, 3305, 245, 647, 306, 147, 112, 103, 672, 69, 317, 61, 224, 71, 52, 479, 62, 106, 166, 215, 132, 137, 321, 998, 427, 846, 787, 542, 1054, 1429, 615, 697, 580, 642, 768, 1244, 462, 4107, 1701, 2394, 4954, 4869, 1841, 1807, 1032, 3075, 331, 488, 627, 1281, 233, 1010, 1178, 727, 830, 1619, 728, 1428, 1849, 4826, 351, 745, 320, 888, 335, 741, 1151, 734, 689, 2143, 1130, 2482, 3609, 4779, 5678, 4186, 2654, 1668, 1290, 702, 1093, 476, 438, 445, 271, 98, 368, 226, 90, 75, 26, 33, 62, 16, 824, 21, 37, 34, 24, 54, 42, 101, 112, 18, 24, 17, 15, 3, 50, 7, 6, 54, 3, 58, 9, 10, 66, 12, 11, 10, 6, 25, 11, 7, 172, 13, 18, 21, 9, 8, 9, 42, 16, 15, 6, 12, 17, 7, 591, 6, 7, 14, 24, 7, 7, 19, 87, 18, 8, 9, 9, 35, 55, 4, 17, 10, 18, 22, 46, 8, 852, 15, 14, 12, 11, 9, 3, 50, 163, 12, 4, 18, 129, 6, 35, 47, 14, 18, 150, 21, 46, 24, 0
Histogram 2
8082, 4857, 1494, 2530, 1604, 1636, 1651, 1681, 1630, 1667, 1636, 1649, 1934, 1775, 1701, 1691, 1478, 1649, 1449, 1449, 1503, 1475, 1497, 1398, 1509, 1747, 1301, 1539, 1575, 1496, 1754, 1432, 1759, 1786, 1679, 1816, 2435, 1174, 1780, 1344, 1749, 2026, 1779, 1742, 1722, 1835, 2306, 1662, 1965, 1885, 2212, 2139, 1930, 2306, 2707, 2289, 2307, 2082, 2360, 2216, 2480, 2243, 2222, 1824, 4555, 1918, 2116, 2275, 2615, 2240, 2703, 2481, 2626, 2708, 3008, 2696, 2561, 2906, 3625, 2419, 3137, 2793, 2747, 2861, 2774, 4124, 3155, 3243, 3523, 3432, 3277, 3456, 2984, 2902, 2819, 2778, 3158, 2997, 2591, 2717, 2553, 2464, 3657, 2296, 2352, 2046, 2124, 1965, 2014, 2096, 1664, 1373, 1607, 1322, 1272, 1113, 1156, 1055, 924, 881, 1019, 669, 929, 636, 590, 463, 524, 177, 1267, 378, 409, 413, 415, 435, 385, 379, 267, 413, 266, 282, 499, 194, 360, 199, 337, 92, 986, 183, 160, 230, 124, 213, 188, 334, 164, 159, 130, 143, 135, 331, 25, 118, 114, 98, 74, 301, 92, 119, 94, 72, 192, 38, 64, 100, 138, 30, 98, 65, 226, 23, 46, 78, 78, 61, 55, 234, 26, 36, 95, 31, 49, 214, 25, 34, 58, 37, 101, 20, 41, 34, 150, 16, 50, 25, 53, 18, 30, 67, 27, 36, 42, 23, 60, 12, 21, 36, 12, 45, 21, 58, 53, 18, 51, 16, 25, 9, 24, 15, 18, 30, 33, 20, 19, 12, 23, 16, 14, 21, 14, 10, 20, 13, 12, 9, 6, 9, 7, 10, 7, 2, 0, 0, 0, 0, 0, 2087
Histogram 3
50, 226, 857, 2018, 1810, 1795, 1840, 1929, 1942, 1693, 1699, 1547, 1564, 1556, 1451, 1439, 1448, 1357, 1428, 1419, 1383, 1705, 1670, 1777, 1826, 1865, 1897, 1924, 2003, 1973, 1813, 1801, 1827, 1696, 1717, 1654, 1678, 1705, 1621, 1523, 1494, 1559, 1434, 1370, 1358, 1385, 1348, 1380, 1368, 1367, 1389, 1445, 1514, 1471, 1465, 1461, 1475, 1484, 1390, 1403, 1324, 1339, 1426, 1432, 1487, 1460, 1469, 1460, 1546, 1504, 1425, 1373, 1391, 1391, 1382, 1311, 1368, 1354, 1325, 1323, 1263, 1325, 1363, 1357, 1325, 1322, 1429, 1419, 1412, 1371, 1266, 1179, 1166, 1076, 1100, 1083, 1103, 1053, 1116, 1080, 1071, 1025, 1088, 1060, 1011, 984, 958, 959, 954, 937, 982, 950, 1001, 963, 965, 875, 1010, 954, 990, 894, 959, 972, 963, 1101, 971, 1042, 1064, 1075, 1029, 1088, 1090, 1068, 1073, 1058, 1102, 1105, 1009, 1062, 1005, 1048, 973, 998, 1034, 1013, 961, 1006, 983, 948, 1031, 972, 952, 1013, 954, 964, 970, 881, 887, 967, 941, 928, 994, 1019, 1106, 1056, 1113, 1071, 1158, 1108, 1178, 1071, 1080, 1074, 1050, 1076, 1106, 1048, 973, 1042, 997, 1034, 934, 863, 935, 845, 839, 803, 764, 782, 787, 771, 766, 751, 745, 804, 789, 765, 681, 658, 690, 672, 650, 635, 695, 619, 572, 499, 535, 565, 564, 520, 516, 568, 530, 479, 507, 424, 446, 455, 380, 395, 371, 360, 391, 373, 351, 388, 426, 349, 417, 421, 400, 443, 470, 485, 456, 495, 452, 484, 457, 518, 519, 631, 652, 693, 762, 771, 807, 906, 991, 1138, 1433, 1545, 2467, 4907, 6743, 1921
smooth histogram
this filter out small local min max and noise. Use symmetric smoothing to avoid shifting to one side. I smooth from left then from the right which lower the shifting a lot.
find/count the local max peaks
Count only big enough peaks (by some treshold). If peak count is not 2 then it is not a bimodal histogram unless you have different definition of bimodal like:
noise count too
no matter how many peaks but single gap must be present
It depends on what for the histograms are used
Here is some code in C++ I busted for this:
void histograms(Graphics::TBitmap *bmp,int xs,int ys,int **pyx)
{
// clear buffer
bmp->Canvas->Brush->Color=clBlack;
bmp->Canvas->FillRect(TRect(0,0,xs,ys));
bmp->Canvas->Font->Color=clAqua;
bmp->Canvas->TextOutA( 5,5,"Raw histogram");
bmp->Canvas->TextOutA(285,5,"Smoothed histogram");
int his1[256]={ 8029, 41, 82, 177, 135, 255, 315, 591, 949, 456, 499, 688, 446, 733, 712, 1595, 2633, 3945, 6134, 9755, 9236, 11911, 11888, 9450, 13119, 8819, 5991, 4399, 6745, 2017, 3747, 1777, 2946, 1623, 2151, 454, 3015, 3176, 2211, 1080, 391, 580, 750, 473, 10424, 334, 559, 621, 340, 2794, 1094, 5274, 2822, 204, 389, 728, 268, 15, 1060, 58, 113, 2728, 52, 3166, 11, 103, 522, 107, 351, 97, 66, 565, 315, 444, 3305, 245, 647, 306, 147, 112, 103, 672, 69, 317, 61, 224, 71, 52, 479, 62, 106, 166, 215, 132, 137, 321, 998, 427, 846, 787, 542, 1054, 1429, 615, 697, 580, 642, 768, 1244, 462, 4107, 1701, 2394, 4954, 4869, 1841, 1807, 1032, 3075, 331, 488, 627, 1281, 233, 1010, 1178, 727, 830, 1619, 728, 1428, 1849, 4826, 351, 745, 320, 888, 335, 741, 1151, 734, 689, 2143, 1130, 2482, 3609, 4779, 5678, 4186, 2654, 1668, 1290, 702, 1093, 476, 438, 445, 271, 98, 368, 226, 90, 75, 26, 33, 62, 16, 824, 21, 37, 34, 24, 54, 42, 101, 112, 18, 24, 17, 15, 3, 50, 7, 6, 54, 3, 58, 9, 10, 66, 12, 11, 10, 6, 25, 11, 7, 172, 13, 18, 21, 9, 8, 9, 42, 16, 15, 6, 12, 17, 7, 591, 6, 7, 14, 24, 7, 7, 19, 87, 18, 8, 9, 9, 35, 55, 4, 17, 10, 18, 22, 46, 8, 852, 15, 14, 12, 11, 9, 3, 50, 163, 12, 4, 18, 129, 6, 35, 47, 14, 18, 150, 21, 46, 24, 0 };
int his2[256]={ 8082, 4857, 1494, 2530, 1604, 1636, 1651, 1681, 1630, 1667, 1636, 1649, 1934, 1775, 1701, 1691, 1478, 1649, 1449, 1449, 1503, 1475, 1497, 1398, 1509, 1747, 1301, 1539, 1575, 1496, 1754, 1432, 1759, 1786, 1679, 1816, 2435, 1174, 1780, 1344, 1749, 2026, 1779, 1742, 1722, 1835, 2306, 1662, 1965, 1885, 2212, 2139, 1930, 2306, 2707, 2289, 2307, 2082, 2360, 2216, 2480, 2243, 2222, 1824, 4555, 1918, 2116, 2275, 2615, 2240, 2703, 2481, 2626, 2708, 3008, 2696, 2561, 2906, 3625, 2419, 3137, 2793, 2747, 2861, 2774, 4124, 3155, 3243, 3523, 3432, 3277, 3456, 2984, 2902, 2819, 2778, 3158, 2997, 2591, 2717, 2553, 2464, 3657, 2296, 2352, 2046, 2124, 1965, 2014, 2096, 1664, 1373, 1607, 1322, 1272, 1113, 1156, 1055, 924, 881, 1019, 669, 929, 636, 590, 463, 524, 177, 1267, 378, 409, 413, 415, 435, 385, 379, 267, 413, 266, 282, 499, 194, 360, 199, 337, 92, 986, 183, 160, 230, 124, 213, 188, 334, 164, 159, 130, 143, 135, 331, 25, 118, 114, 98, 74, 301, 92, 119, 94, 72, 192, 38, 64, 100, 138, 30, 98, 65, 226, 23, 46, 78, 78, 61, 55, 234, 26, 36, 95, 31, 49, 214, 25, 34, 58, 37, 101, 20, 41, 34, 150, 16, 50, 25, 53, 18, 30, 67, 27, 36, 42, 23, 60, 12, 21, 36, 12, 45, 21, 58, 53, 18, 51, 16, 25, 9, 24, 15, 18, 30, 33, 20, 19, 12, 23, 16, 14, 21, 14, 10, 20, 13, 12, 9, 6, 9, 7, 10, 7, 2, 0, 0, 0, 0, 0, 2087 };
int his3[256]={ 50, 226, 857, 2018, 1810, 1795, 1840, 1929, 1942, 1693, 1699, 1547, 1564, 1556, 1451, 1439, 1448, 1357, 1428, 1419, 1383, 1705, 1670, 1777, 1826, 1865, 1897, 1924, 2003, 1973, 1813, 1801, 1827, 1696, 1717, 1654, 1678, 1705, 1621, 1523, 1494, 1559, 1434, 1370, 1358, 1385, 1348, 1380, 1368, 1367, 1389, 1445, 1514, 1471, 1465, 1461, 1475, 1484, 1390, 1403, 1324, 1339, 1426, 1432, 1487, 1460, 1469, 1460, 1546, 1504, 1425, 1373, 1391, 1391, 1382, 1311, 1368, 1354, 1325, 1323, 1263, 1325, 1363, 1357, 1325, 1322, 1429, 1419, 1412, 1371, 1266, 1179, 1166, 1076, 1100, 1083, 1103, 1053, 1116, 1080, 1071, 1025, 1088, 1060, 1011, 984, 958, 959, 954, 937, 982, 950, 1001, 963, 965, 875, 1010, 954, 990, 894, 959, 972, 963, 1101, 971, 1042, 1064, 1075, 1029, 1088, 1090, 1068, 1073, 1058, 1102, 1105, 1009, 1062, 1005, 1048, 973, 998, 1034, 1013, 961, 1006, 983, 948, 1031, 972, 952, 1013, 954, 964, 970, 881, 887, 967, 941, 928, 994, 1019, 1106, 1056, 1113, 1071, 1158, 1108, 1178, 1071, 1080, 1074, 1050, 1076, 1106, 1048, 973, 1042, 997, 1034, 934, 863, 935, 845, 839, 803, 764, 782, 787, 771, 766, 751, 745, 804, 789, 765, 681, 658, 690, 672, 650, 635, 695, 619, 572, 499, 535, 565, 564, 520, 516, 568, 530, 479, 507, 424, 446, 455, 380, 395, 371, 360, 391, 373, 351, 388, 426, 349, 417, 421, 400, 443, 470, 485, 456, 495, 452, 484, 457, 518, 519, 631, 652, 693, 762, 771, 807, 906, 991, 1138, 1433, 1545, 2467, 4907, 6743, 1921 };
int *his,tmp[256],a,x0,y0,x,y,h,tr=12,sm=10,peak[256],peaks;
// loop through histograms
for (y0=20,h=0;;h++)
{
x0=5;if (h==0) his=his1;
else if (h==1) his=his2;
else if (h==2) his=his2;
else break;
// rescale his <0,?> to tmp <0-100>
for (y=his[0],x=0;x<256;x++) if (y<his[x]) y=his[x]; // y=max
for ( x=0;x<256;x++) tmp[x]=(his[x]*100)/y;
// draw tmp
for (x=0;x<256;x++) for (pyx[y0+100][x0+x]=0x00004040,y=0;y<tmp[x];y++) pyx[y0+100-y][x0+x]=(40+x)*0x00010101;
x0+=280;
// smooth tmp few times
for (y=0;y<sm;y++)
{
// from both directions to avoid shifting to one side
for (x=0;x<255;x++) tmp[x]=((90*tmp[x])+(10*tmp[x+1]))/100;
for (x=255;x>0;x--) tmp[x]=((90*tmp[x])+(10*tmp[x-1]))/100;
}
// find (count) peaks
for (peaks=0,a=0,y=0,x=0;x<255;x++)
{
if (tmp[x]<tmp[x+1]){ if ((y< 0)&&(a-tmp[x]>tr)){ a=tmp[x]; } y=+1; }
else if (tmp[x]>tmp[x+1]){ if ((y>=0)&&(tmp[x]-a>tr)){ a=tmp[x]; peak[peaks]=x; peaks++; } y=-1; }
}
// draw tmp
for (x=0;x<256;x++) for (pyx[y0+100][x0+x]=0x00004040,y=0;y<tmp[x];y++) pyx[y0+100-y][x0+x]=(40+x)*0x00010101;
// draw peaks
bmp->Canvas->Pen->Color=clAqua;
bmp->Canvas->Brush->Color=clAqua;
for (a=0;a<peaks;a++)
{
x=x0+peak[a];
y=y0+100-tmp[peak[a]];
bmp->Canvas->Ellipse(x-5,y-5,x+5,y+5);
}
// draw cross for not bimodal histograms
if (peaks!=2)
{
bmp->Canvas->Pen->Color=clRed;
bmp->Canvas->MoveTo(x0 ,y0 );
bmp->Canvas->LineTo(x0+256,y0+100);
bmp->Canvas->MoveTo(x0+256,y0 );
bmp->Canvas->LineTo(x0 ,y0+100);
}
y0+=128;
}
}
you can ignore the pyx[y][x] and bmp-> stuff it is just rendering
pyx[y][x] is direct 32bit pixel access of bitmap bmp
and bmp->Canvas is VCL encapsulated Windows GDI interface of bitmap bmp
xs,ys is bitmap resolution
set treshold tr and smooth sm to suite your needs best
If you have too much different types of histogram then you need to apply dynamic tresholding or different approach for peak finding this is how it looks like for your histograms:
Where Histogram 1 is the top one. Hope the code is clear enough if not comment me... if you rescale to power of 2 instead of 100 then you can change the multiplications and divisions to bit shifts to speed this a bit. I choose 100 for more clear selection of tresholds and smoothing coefficients...
I don't think there is a simple and straightforward solution for that.
If you think each peak in your histogram as a cluster, you can try to implement some sort of clustering algorithm that is able to automatically detect the number of peaks in your histogram.
You can start looking at here:
Determining the number of clusters in a data set
Kmeans without knowing the number of clusters?
AUTOMATIC DETERMINATION OF THE NUMBER OF CLUSTERS USING SPECTRAL
ALGORITHMS

How to programmatically monitor if a docker container exited?

I am running multiple named docker containers (200+) on my VM Host.
I have a manager script/code that is supposed to manage the containers from the host.
I would like to know if there is any event-based mechanism to get notified when a container stops/fails. So that I can restart the stopped container.
One solution I could think of is doing a periodic docker inspect and looking at State.Pid or State.Running to confirm the status.
But,instead of periodic polling, it would be better if the manager is notified with pid/name when a container fails so that, the particular container alone can be restarted.
On a general note, are there ways to programmatically monitor the status of a process from a different process that is not the parent ?
Look at docker events - there is an event for container 'die'.
There is also an http interface to get the same information programmatically - see here
You may want to do a web search for 'docker orchestration' - many projects springing up to manage multiple containers in the way you describe.
If you just want to restart the containers why don't you use a restart policy?
docker run --restart=always IMAGE
psutil seems to do what you want
http://pypi.python.org/pypi/psutil
From Python
import psutil
psutil.pids()
[1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 51, 52, 53, 54, 56, 57, 58, 59, 61, 62, 63, 64, 65, 66, 67, 69, 70, 71, 72, 73, 74, 76, 77, 78, 79, 80, 81, 82, 94, 97, 98, 117, 118, 137, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 200, 201, 210, 211, 213, 214, 229, 230, 411, 416, 419, 526, 527, 542, 543, 544, 545, 555, 569, 625, 709, 714, 756, 781, 782, 796, 862, 863, 864, 869, 914, 944, 945, 948, 954, 996, 1052, 1061, 1064, 1067, 1170, 1174, 1179, 1180, 1183, 1234, 1240, 1241, 1245, 1323, 1328, 1340, 1351, 1354, 1390, 1408, 1457, 1507, 1531, 1631, 1662, 1933, 1972, 1981, 1987, 1989, 1993, 2346, 2348, 2413, 2422, 2429, 2442, 2445, 2449, 2451, 2457, 2461, 2471, 2489, 2490, 2491, 2493, 2497, 2501, 2505, 2509, 2513, 2524, 2546, 2549, 2551, 2554, 2563, 2567, 2572, 2573, 2576, 2578, 2586, 2595, 2598, 2624, 2644, 2655, 2665, 2667, 2687, 2689, 2693, 2699, 2744, 2752, 2785, 2789, 2794, 2798, 2804, 2817, 2820, 2830, 2838, 2856, 2862, 2864, 2886, 2903, 2935, 2972, 2985, 2986, 3138, 3164, 3211, 3368, 3371, 3557, 4125, 4352, 4443, 4444, 4743, 4818, 4819, 4840, 4841, 4844, 4845, 4866, 4876, 6142, 6363, 6366, 6372, 6378, 6385, 6391, 6452, 6518, 6524, 6531, 6555, 6558, 6601]
p = psutil.Process(2862)
p.status()
'sleeping'

Avoid generating N+1 queries while fetching an object graph through ActiveRecord

I have two Models: AssetItem and Title.
An AssetItem belongs to a Title / Title has many asset_items.
A Title can have a parent Title (thereby creating a hierarchy); this parent/child relationship within Titles is managed by the ancestry gem.
I've got a controller that is fetching a bunch of AssetItems. Those AssetItems should come with their Title, that Title's parent, and that Title's parent and so on. In the end, the controller presents this object graph in JSON. I am trying to find a way to eager-load all of the title hierarchy for a given asset item.
I am doing this in my controller:
def show
asset_items = AssetItem.includes(:title).where(:id => list_of_ids)
eager_load_titles_for(asset_items)
respond_with asset_items, serializer: AssetItemSerializer
end
private
def eager_load_titles_for(asset_items)
title_ids = asset_items.map { |a| a.title.ancestor_ids }.flatten
Title.find(title_ids) # these titles are not really used, this is just to eager load them
end
The idea is, that a collection of asset items are fetched and then the title ids associated with those asset items are simply loaded (but not used), so that when the serializer does something like object.parent.parent.parent.some_attribute, it gets that Title from the cache, not the db.
However, the results in the log do not match my expectations:
10:19:27 web.1 | AssetItem Load (1.3ms) SELECT "asset_items".* FROM "asset_items" WHERE "asset_items"."id" IN (291, 311, 316, 309, 323, 304, 296, 289, 328, 284, 373, 342, 330, 335, 361, 366, 354, 347, 359, 380, 400, 405, 392, 378, 397, 417, 412, 385, 436, 429, 450, 448, 431, 455, 462, 424, 443, 481, 498, 467, 501, 493, 474, 506, 486, 479, 532, 525, 520, 513)
10:19:27 web.1 | Title Load (0.6ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" IN (167, 182, 188, 203, 227, 242, 248, 263, 284, 299, 305, 320, 341, 356, 377, 392, 398, 413, 434, 449, 455, 470, 491, 506, 515, 530, 551, 566, 587, 602, 608, 623, 644, 659, 665, 680, 701, 716, 737, 752, 758, 773, 794, 809, 818, 833, 854, 875, 890, 911)
10:19:27 web.1 | Title Load (0.9ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" IN (186, 187, 246, 247, 261, 262, 240, 241, 282, 283, 225, 226, 201, 202, 180, 181, 297, 298, 165, 166, 432, 433, 339, 340, 303, 304, 318, 319, 396, 397, 411, 412, 375, 376, 354, 355, 390, 391, 453, 454, 513, 514, 528, 529, 489, 490, 447, 448, 504, 505, 564, 565, 549, 550, 468, 469, 621, 622, 600, 601, 663, 664, 657, 658, 606, 607, 678, 679, 699, 700, 585, 586, 642, 643, 756, 757, 807, 808, 714, 715, 816, 817, 792, 793, 735, 736, 831, 832, 771, 772, 750, 751, 909, 910, 888, 889, 873, 874, 852, 853)
10:19:27 web.1 | CACHE (0.0ms) SELECT "asset_items".* FROM "asset_items" WHERE "asset_items"."id" IN (291, 311, 316, 309, 323, 304, 296, 289, 328, 284, 373, 342, 330, 335, 361, 366, 354, 347, 359, 380, 400, 405, 392, 378, 397, 417, 412, 385, 436, 429, 450, 448, 431, 455, 462, 424, 443, 481, 498, 467, 501, 493, 474, 506, 486, 479, 532, 525, 520, 513)
10:19:27 web.1 | CACHE (0.0ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" IN (167, 182, 188, 203, 227, 242, 248, 263, 284, 299, 305, 320, 341, 356, 377, 392, 398, 413, 434, 449, 455, 470, 491, 506, 515, 530, 551, 566, 587, 602, 608, 623, 644, 659, 665, 680, 701, 716, 737, 752, 758, 773, 794, 809, 818, 833, 854, 875, 890, 911)
10:19:27 web.1 | CACHE (0.0ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" IN (186, 187, 246, 247, 261, 262, 240, 241, 282, 283, 225, 226, 201, 202, 180, 181, 297, 298, 165, 166, 432, 433, 339, 340, 303, 304, 318, 319, 396, 397, 411, 412, 375, 376, 354, 355, 390, 391, 453, 454, 513, 514, 528, 529, 489, 490, 447, 448, 504, 505, 564, 565, 549, 550, 468, 469, 621, 622, 600, 601, 663, 664, 657, 658, 606, 607, 678, 679, 699, 700, 585, 586, 642, 643, 756, 757, 807, 808, 714, 715, 816, 817, 792, 793, 735, 736, 831, 832, 771, 772, 750, 751, 909, 910, 888, 889, 873, 874, 852, 853)
10:19:27 web.1 | Title Load (0.4ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" = $1 LIMIT 1 [["id", 186]]
10:19:27 web.1 | Title Load (0.2ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" = $1 LIMIT 1 [["id", 187]]
10:19:27 web.1 | CACHE (0.0ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" = $1 LIMIT 1 [["id", 187]]
10:19:27 web.1 | CACHE (0.0ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" = $1 LIMIT 1 [["id", 187]]
10:19:27 web.1 | CACHE (0.0ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" = $1 LIMIT 1 [["id", 187]]
10:19:27 web.1 | CACHE (0.0ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" = $1 LIMIT 1 [["id", 187]]
10:19:27 web.1 | CACHE (0.0ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" = $1 LIMIT 1 [["id", 187]]
As you can see, initially title ID 186, and 187 are loaded in the where clause here:
10:19:27 web.1 | Title Load (0.9ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" IN (186, 187, 246, 247, 261, 262, 240, 241, 282, 283, 225, 226, 201, 202, 180, 181, 297, 298, 165, 166, 432, 433, 339, 340, 303, 304, 318, 319, 396, 397, 411, 412, 375, 376, 354, 355, 390, 391, 453, 454, 513, 514, 528, 529, 489, 490, 447, 448, 504, 505, 564, 565, 549, 550, 468, 469, 621, 622, 600, 601, 663, 664, 657, 658, 606, 607, 678, 679, 699, 700, 585, 586, 642, 643, 756, 757, 807, 808, 714, 715, 816, 817, 792, 793, 735, 736, 831, 832, 771, 772, 750, 751, 909, 910, 888, 889, 873, 874, 852, 853)
and then below that, we see again:
10:19:27 web.1 | Title Load (0.4ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" = $1 LIMIT 1 [["id", 186]]
10:19:27 web.1 | Title Load (0.2ms) SELECT "titles".* FROM "titles" WHERE "titles"."id" = $1 LIMIT 1 [["id", 187]]
And that is why I am wondering what I am doing wrong. Any help would be greatly appreciated.

Mahout gives extremely low hits/precision when running against Movielens 1m data

Problem Description:
I'm using Mahout GeneircItemBasedRecommender running on Movielens 1m data, but always get 0 hits against testing data set, with the precision result is merely 0.056528926, which is neither unreasonable nor unacceptable. On the other hand, when using GenericUserBasedRecommender, I can get preccision: 0.13883367, why is it happening? I can bearly figure out why there is such a large difference between item-based and user-based algorithm, is there any wrong in my code? I've read Mahout Precision Recall Movielens 10M, but seems it didn't answer my question.
Summary:
Item-Based Recommender gives extreamly low hits/precision, while
User-Based's hits/precision seems make sense
Item-Based Recommender recommended items stick around 0~500, while User-Based's recommended item spectrum seems make sense
Item-Based Recommender gives ALL recommended item a prediction value of 5, while User-Based's prediction value have aboud 1/9 below
SlopeOne Recommender acts similar with Item-Based Recommender
Note: the result is given when splitting data by 5:5 (2-fold cross validation)
Attachements:
java code
results log
mahout pom file
Please help!!!!
Thanks in advance!!!
Java Code:
import com.bestv.mahout.deprecated.SimilarityFactory;
import org.apache.mahout.cf.taste.common.TasteException;
import org.apache.mahout.cf.taste.common.Weighting;
import org.apache.mahout.cf.taste.eval.IRStatistics;
import org.apache.mahout.cf.taste.eval.RecommenderBuilder;
import org.apache.mahout.cf.taste.eval.RecommenderEvaluator;
import org.apache.mahout.cf.taste.eval.RecommenderIRStatsEvaluator;
import org.apache.mahout.cf.taste.impl.common.LongPrimitiveIterator;
import org.apache.mahout.cf.taste.impl.eval.GenericRecommenderIRStatsEvaluator;
import org.apache.mahout.cf.taste.impl.eval.RMSRecommenderEvaluator;
import org.apache.mahout.cf.taste.impl.model.file.FileDataModel;
import org.apache.mahout.cf.taste.impl.neighborhood.NearestNUserNeighborhood;
import org.apache.mahout.cf.taste.impl.recommender.GenericItemBasedRecommender;
import org.apache.mahout.cf.taste.impl.recommender.GenericUserBasedRecommender;
import org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommender;
import org.apache.mahout.cf.taste.impl.similarity.PearsonCorrelationSimilarity;
import org.apache.mahout.cf.taste.model.DataModel;
import org.apache.mahout.cf.taste.neighborhood.UserNeighborhood;
import org.apache.mahout.cf.taste.recommender.RecommendedItem;
import org.apache.mahout.cf.taste.recommender.Recommender;
import org.apache.mahout.cf.taste.similarity.ItemSimilarity;
import org.apache.mahout.cf.taste.similarity.UserSimilarity;
import org.apache.mahout.common.RandomUtils;
import org.junit.Test;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
import java.util.List;
public class RecommenderEvaluatorTest {
#Test
public void testSelfItemBasedIREvaluator() throws IOException, TasteException {
DataModel trainDataModal = new FileDataModel(new File("/cmsdata/cflab/train/training.csv"));
DataModel testDataModal = new FileDataModel(new File("/cmsdata/cflab/test/testing.csv"));
ItemSimilarity perasonSimilarity = new PearsonCorrelationSimilarity(trainDataModal);
Recommender pearsonItemRecommender = new GenericItemBasedRecommender(trainDataModal, perasonSimilarity);
long hitCnt = 0;
long totalCnt = 0;
LongPrimitiveIterator userIDItr = testDataModal.getUserIDs();
while (userIDItr.hasNext()) {
List<Long> recIDList = new ArrayList<Long>();
List<Long> realIDList = new ArrayList<Long>();
Long uid = userIDItr.nextLong();
List<RecommendedItem> recItemList = pearsonItemRecommender.recommend(uid, 10);
for (RecommendedItem item : recItemList) {
recIDList.add(item.getItemID());
}
long[] tempArr = testDataModal.getPreferencesFromUser(uid).getIDs();
if (tempArr != null) {
for (int i=0; i<tempArr.length; i++) {
realIDList.add(new Long(tempArr[i]));
}
}
totalCnt += recIDList.size();
System.out.println("------------------ userid: " + uid + "------------------");
System.out.println(recIDList.toString());
System.out.println(realIDList.toString());
recIDList.retainAll(realIDList);
hitCnt += recIDList.size();
if (recIDList.size()>0)
System.out.println("************ current hits: " + recIDList.size() + " ***************");
System.out.println("=============== precision: " + (float)hitCnt/totalCnt + " ======================");
System.out.println(recItemList);
}
}
#Test
public void testSelfUserBasedIREvaluator() throws IOException, TasteException {
DataModel trainDataModal = new FileDataModel(new File("/cmsdata/cflab/train/training.csv"));
DataModel testDataModal = new FileDataModel(new File("/cmsdata/cflab/test/testing.csv"));
UserSimilarity perasonSimilarity = new PearsonCorrelationSimilarity(trainDataModal);
NearestNUserNeighborhood neighborhood = new NearestNUserNeighborhood(20, perasonSimilarity, trainDataModal);
Recommender pearsonItemRecommender = new GenericUserBasedRecommender(trainDataModal, neighborhood, perasonSimilarity);
long hitCnt = 0;
long totalCnt = 0;
LongPrimitiveIterator userIDItr = testDataModal.getUserIDs();
while (userIDItr.hasNext()) {
List<Long> recIDList = new ArrayList<Long>();
List<Long> realIDList = new ArrayList<Long>();
Long uid = userIDItr.nextLong();
List<RecommendedItem> recItemList = pearsonItemRecommender.recommend(uid, 10);
for (RecommendedItem item : recItemList) {
recIDList.add(item.getItemID());
}
long[] tempArr = testDataModal.getPreferencesFromUser(uid).getIDs();
if (tempArr != null) {
for (int i=0; i<tempArr.length; i++) {
realIDList.add(new Long(tempArr[i]));
}
}
totalCnt += recIDList.size();
System.out.println("------------------ userid: " + uid + "------------------");
System.out.println(recIDList.toString());
System.out.println(realIDList.toString());
recIDList.retainAll(realIDList);
hitCnt += recIDList.size();
if (recIDList.size()>0)
System.out.println("************ current hits: " + recIDList.size() + " *****************");
System.out.println("=============== precision: " + (float)hitCnt/totalCnt + " =================");
System.out.println(recItemList);
}
}
#Test
public void testSelfSlopeOneIREvaluator() throws IOException, TasteException {
DataModel trainDataModal = new FileDataModel(new File("/cmsdata/cflab/train/training.csv"));
DataModel testDataModal = new FileDataModel(new File("/cmsdata/cflab/test/testing.csv"));
Recommender slopeOneRecommender = new SlopeOneRecommender(trainDataModal);
long hitCnt = 0;
long totalCnt = 0;
LongPrimitiveIterator userIDItr = testDataModal.getUserIDs();
while (userIDItr.hasNext()) {
List<Long> recIDList = new ArrayList<Long>();
List<Long> realIDList = new ArrayList<Long>();
Long uid = userIDItr.nextLong();
List<RecommendedItem> recItemList = slopeOneRecommender.recommend(uid, 10);
for (RecommendedItem item : recItemList) {
recIDList.add(item.getItemID());
}
long[] tempArr = testDataModal.getPreferencesFromUser(uid).getIDs();
if (tempArr != null) {
for (int i=0; i<tempArr.length; i++) {
realIDList.add(new Long(tempArr[i]));
}
}
totalCnt += recIDList.size();
System.out.println("------------------ userid: " + uid + "------------------");
System.out.println(recIDList.toString());
System.out.println(realIDList.toString());
recIDList.retainAll(realIDList);
hitCnt += recIDList.size();
if (recIDList.size()>0)
System.out.println("************ current hits: " + recIDList.size() + " *****************");
System.out.println("=============== precision: " + (float)hitCnt/totalCnt + " =================");
System.out.println(recItemList);
}
}
}
ItemBased log (user 1-20, there's hardly hit record)
------------------ userid: 1------------------
[84, 81, 61, 71, 75, 15, 7, 57, 64, 26]
[1, 48, 150, 260, 531, 588, 594, 595, 661, 783, 914, 938, 1022, 1029, 1035, 1207, 1246, 1270, 1545, 1566, 1962, 2028, 2294, 2321, 2340, 2398, 2762, 2791, 2804]
=============== precision: 0.0 ======================
[RecommendedItem[item:84, value:5.0], RecommendedItem[item:81, value:5.0], RecommendedItem[item:61, value:5.0], RecommendedItem[item:71, value:5.0], RecommendedItem[item:75, value:5.0], RecommendedItem[item:15, value:5.0], RecommendedItem[item:7, value:5.0], RecommendedItem[item:57, value:5.0], RecommendedItem[item:64, value:5.0], RecommendedItem[item:26, value:5.0]]
------------------ userid: 2------------------
[216, 166, 117, 183, 190, 37, 106, 128, 97, 33]
[21, 95, 163, 165, 235, 292, 318, 349, 356, 434, 457, 459, 590, 593, 647, 648, 902, 920, 982, 1084, 1124, 1193, 1198, 1207, 1217, 1244, 1253, 1259, 1293, 1357, 1372, 1527, 1552, 1784, 1834, 1873, 1917, 1955, 1962, 1968, 2002, 2006, 2028, 2067, 2126, 2194, 2278, 2312, 2353, 2396, 2427, 2490, 2501, 2628, 2852, 2858, 3035, 3095, 3257, 3334, 3578, 3678, 3809, 3893]
=============== precision: 0.0 ======================
[RecommendedItem[item:216, value:5.0], RecommendedItem[item:166, value:5.0], RecommendedItem[item:117, value:5.0], RecommendedItem[item:183, value:5.0], RecommendedItem[item:190, value:5.0], RecommendedItem[item:37, value:5.0], RecommendedItem[item:106, value:5.0], RecommendedItem[item:128, value:5.0], RecommendedItem[item:97, value:5.0], RecommendedItem[item:33, value:5.0]]
------------------ userid: 3------------------
[182, 174, 114, 97, 162, 82, 78, 8, 55, 59]
[260, 480, 552, 1079, 1197, 1198, 1210, 1259, 1261, 1270, 1580, 2617, 2735, 2858, 2997, 3421, 3534, 3619, 3671]
=============== precision: 0.0 ======================
[RecommendedItem[item:182, value:5.0], RecommendedItem[item:174, value:5.0], RecommendedItem[item:114, value:5.0], RecommendedItem[item:97, value:5.0], RecommendedItem[item:162, value:5.0], RecommendedItem[item:82, value:5.0], RecommendedItem[item:78, value:5.0], RecommendedItem[item:8, value:5.0], RecommendedItem[item:55, value:5.0], RecommendedItem[item:59, value:5.0]]
------------------ userid: 4------------------
[87, 86, 72, 55, 75, 40, 46, 28, 12, 30]
[1210, 1214, 1240, 2028, 2951, 3468, 3527]
=============== precision: 0.0 ======================
[RecommendedItem[item:87, value:5.0], RecommendedItem[item:86, value:5.0], RecommendedItem[item:72, value:5.0], RecommendedItem[item:55, value:5.0], RecommendedItem[item:75, value:5.0], RecommendedItem[item:40, value:5.0], RecommendedItem[item:46, value:5.0], RecommendedItem[item:28, value:5.0], RecommendedItem[item:12, value:5.0], RecommendedItem[item:30, value:5.0]]
------------------ userid: 5------------------
[340, 324, 321, 258, 287, 87, 263, 128, 33, 234]
[6, 24, 39, 41, 47, 52, 224, 272, 288, 296, 318, 321, 356, 377, 497, 509, 515, 800, 866, 896, 913, 994, 1127, 1171, 1175, 1192, 1243, 1279, 1392, 1449, 1466, 1500, 1529, 1535, 1610, 1643, 1649, 1650, 1683, 1684, 1715, 1719, 1729, 1730, 1734, 1747, 1788, 1909, 1912, 1966, 2028, 2058, 2188, 2268, 2318, 2337, 2355, 2359, 2390, 2395, 2427, 2428, 2580, 2683, 2700, 2716, 2721, 2723, 2762, 2858, 2952, 2987, 2997, 3006, 3016, 3020, 3079, 3083, 3260, 3386, 3408, 3409, 3476, 3513, 3578, 3624, 3744, 3786, 3793, 3799]
************ current hits: 1 ***************
=============== precision: 0.02 ======================
[RecommendedItem[item:340, value:5.0], RecommendedItem[item:324, value:5.0], RecommendedItem[item:321, value:5.0], RecommendedItem[item:258, value:5.0], RecommendedItem[item:287, value:5.0], RecommendedItem[item:87, value:5.0], RecommendedItem[item:263, value:5.0], RecommendedItem[item:128, value:5.0], RecommendedItem[item:33, value:5.0], RecommendedItem[item:234, value:5.0]]
------------------ userid: 6------------------
[210, 209, 202, 190, 148, 101, 30, 175, 44, 38]
[199, 266, 368, 377, 383, 588, 595, 597, 920, 1028, 1035, 1101, 1188, 1210, 1296, 1441, 1674, 1947, 1959, 2017, 2100, 2406, 2506, 2802, 2858, 3074, 3501, 3508, 3600, 3624, 3699]
=============== precision: 0.016666668 ======================
[RecommendedItem[item:210, value:5.0], RecommendedItem[item:209, value:5.0], RecommendedItem[item:202, value:5.0], RecommendedItem[item:190, value:5.0], RecommendedItem[item:148, value:5.0], RecommendedItem[item:101, value:5.0], RecommendedItem[item:30, value:5.0], RecommendedItem[item:175, value:5.0], RecommendedItem[item:44, value:5.0], RecommendedItem[item:38, value:5.0]]
------------------ userid: 7------------------
[200, 189, 147, 154, 171, 80, 7, 68, 35, 49]
[6, 110, 349, 457, 474, 480, 589, 1221, 1270, 1573, 1610, 1722, 1997, 2028, 2571, 2916, 3256, 3578, 3753]
=============== precision: 0.014285714 ======================
[RecommendedItem[item:200, value:5.0], RecommendedItem[item:189, value:5.0], RecommendedItem[item:147, value:5.0], RecommendedItem[item:154, value:5.0], RecommendedItem[item:171, value:5.0], RecommendedItem[item:80, value:5.0], RecommendedItem[item:7, value:5.0], RecommendedItem[item:68, value:5.0], RecommendedItem[item:35, value:5.0], RecommendedItem[item:49, value:5.0]]
------------------ userid: 8------------------
[202, 183, 75, 104, 80, 44, 33, 53, 98, 28]
[14, 16, 17, 24, 25, 39, 42, 110, 111, 150, 151, 161, 163, 230, 253, 265, 269, 288, 296, 337, 345, 377, 506, 508, 527, 538, 555, 589, 741, 1059, 1120, 1210, 1213, 1277, 1357, 1393, 1476, 1573, 1580, 1589, 1621, 1639, 1660, 1673, 1678, 1682, 1693, 1704, 1711, 1730, 1916, 2006, 2028, 2278, 2297, 2320, 2396, 2427, 2429, 2571, 2600, 2688, 2692, 2699, 2712, 2858, 3105, 3107, 3147, 3155, 3213, 3246, 3250, 3267, 3386, 3425, 3481]
=============== precision: 0.0125 ======================
[RecommendedItem[item:202, value:5.0], RecommendedItem[item:183, value:5.0], RecommendedItem[item:75, value:5.0], RecommendedItem[item:104, value:5.0], RecommendedItem[item:80, value:5.0], RecommendedItem[item:44, value:5.0], RecommendedItem[item:33, value:5.0], RecommendedItem[item:53, value:5.0], RecommendedItem[item:98, value:5.0], RecommendedItem[item:28, value:5.0]]
------------------ userid: 9------------------
[210, 200, 102, 157, 149, 49, 46, 119, 146, 28]
[1, 25, 50, 223, 318, 349, 367, 377, 412, 428, 457, 508, 529, 590, 593, 778, 805, 920, 1060, 1089, 1210, 1223, 1265, 1307, 1343, 1393, 1466, 1552, 1639, 1682, 1721, 1784, 1921, 1961, 2028, 2268, 2302, 3006, 3147, 3148, 3253, 3255, 3298, 3484, 3510, 3751, 3793]
=============== precision: 0.011111111 ======================
[RecommendedItem[item:210, value:5.0], RecommendedItem[item:200, value:5.0], RecommendedItem[item:102, value:5.0], RecommendedItem[item:157, value:5.0], RecommendedItem[item:149, value:5.0], RecommendedItem[item:49, value:5.0], RecommendedItem[item:46, value:5.0], RecommendedItem[item:119, value:5.0], RecommendedItem[item:146, value:5.0], RecommendedItem[item:28, value:5.0]]
------------------ userid: 10------------------
[297, 264, 291, 244, 263, 119, 103, 37, 97, 33]
[1, 7, 24, 32, 62, 150, 153, 180, 186, 203, 223, 275, 277, 292, 316, 317, 318, 329, 333, 351, 356, 367, 435, 480, 520, 541, 542, 543, 551, 586, 587, 595, 596, 648, 653, 671, 745, 750, 784, 802, 858, 898, 899, 904, 916, 918, 923, 938, 953, 1015, 1035, 1059, 1079, 1084, 1091, 1104, 1127, 1135, 1136, 1148, 1196, 1197, 1198, 1200, 1220, 1221, 1230, 1240, 1250, 1252, 1254, 1256, 1282, 1283, 1291, 1293, 1302, 1304, 1307, 1320, 1339, 1356, 1380, 1394, 1411, 1544, 1566, 1573, 1586, 1654, 1682, 1721, 1756, 1884, 1918, 1921, 1927, 1948, 1961, 1967, 2005, 2021, 2040, 2046, 2047, 2072, 2080, 2082, 2090, 2093, 2096, 2108, 2111, 2124, 2133, 2138, 2161, 2174, 2193, 2294, 2300, 2336, 2355, 2371, 2378, 2398, 2405, 2407, 2423, 2431, 2436, 2470, 2497, 2501, 2529, 2530, 2599, 2617, 2622, 2628, 2640, 2657, 2693, 2717, 2746, 2762, 2770, 2791, 2795, 2804, 2872, 2918, 2946, 2948, 2959, 2968, 2997, 3035, 3037, 3066, 3095, 3114, 3155, 3174, 3175, 3194, 3254, 3309, 3358, 3359, 3438, 3489, 3500, 3591, 3668, 3675, 3699, 3701, 3702, 3703, 3723, 3751, 3809, 3812, 3869, 3928]
=============== precision: 0.01 ======================
[RecommendedItem[item:297, value:5.0], RecommendedItem[item:264, value:5.0], RecommendedItem[item:291, value:5.0], RecommendedItem[item:244, value:5.0], RecommendedItem[item:263, value:5.0], RecommendedItem[item:119, value:5.0], RecommendedItem[item:103, value:5.0], RecommendedItem[item:37, value:5.0], RecommendedItem[item:97, value:5.0], RecommendedItem[item:33, value:5.0]]
UserBased log (user 1-20, there are some hit records)
------------------ userid: 1------------------
[2248, 3479, 1952, 1208, 2966, 457, 1278, 474, 2795, 50]
[1, 48, 150, 260, 531, 588, 594, 595, 661, 783, 914, 938, 1022, 1029, 1035, 1207, 1246, 1270, 1545, 1566, 1962, 2028, 2294, 2321, 2340, 2398, 2762, 2791, 2804]
=============== precision: 0.0 =================
[RecommendedItem[item:2248, value:5.0], RecommendedItem[item:3479, value:5.0], RecommendedItem[item:1952, value:5.0], RecommendedItem[item:1208, value:5.0], RecommendedItem[item:2966, value:5.0], RecommendedItem[item:457, value:5.0], RecommendedItem[item:1278, value:5.0], RecommendedItem[item:474, value:5.0], RecommendedItem[item:2795, value:5.0], RecommendedItem[item:50, value:4.75]]
------------------ userid: 2------------------
[1259, 2501, 527, 2324, 2369, 778, 1206, 1200, 3160, 296]
[21, 95, 163, 165, 235, 292, 318, 349, 356, 434, 457, 459, 590, 593, 647, 648, 902, 920, 982, 1084, 1124, 1193, 1198, 1207, 1217, 1244, 1253, 1259, 1293, 1357, 1372, 1527, 1552, 1784, 1834, 1873, 1917, 1955, 1962, 1968, 2002, 2006, 2028, 2067, 2126, 2194, 2278, 2312, 2353, 2396, 2427, 2490, 2501, 2628, 2852, 2858, 3035, 3095, 3257, 3334, 3578, 3678, 3809, 3893]
************ current hits: 2 *****************
=============== precision: 0.1 =================
[RecommendedItem[item:1259, value:5.0], RecommendedItem[item:2501, value:5.0], RecommendedItem[item:527, value:5.0], RecommendedItem[item:2324, value:5.0], RecommendedItem[item:2369, value:5.0], RecommendedItem[item:778, value:5.0], RecommendedItem[item:1206, value:5.0], RecommendedItem[item:1200, value:5.0], RecommendedItem[item:3160, value:5.0], RecommendedItem[item:296, value:5.0]]
------------------ userid: 3------------------
[2664, 912, 318, 858, 2078, 3361, 296, 253, 223, 3178]
[260, 480, 552, 1079, 1197, 1198, 1210, 1259, 1261, 1270, 1580, 2617, 2735, 2858, 2997, 3421, 3534, 3619, 3671]
=============== precision: 0.06666667 =================
[RecommendedItem[item:2664, value:5.0], RecommendedItem[item:912, value:5.0], RecommendedItem[item:318, value:5.0], RecommendedItem[item:858, value:5.0], RecommendedItem[item:2078, value:5.0], RecommendedItem[item:3361, value:5.0], RecommendedItem[item:296, value:5.0], RecommendedItem[item:253, value:5.0], RecommendedItem[item:223, value:5.0], RecommendedItem[item:3178, value:5.0]]
------------------ userid: 4------------------
[377, 2924, 2700, 1223, 333, 34, 2565, 2571, 58, 50]
[1210, 1214, 1240, 2028, 2951, 3468, 3527]
=============== precision: 0.05 =================
[RecommendedItem[item:377, value:5.0], RecommendedItem[item:2924, value:5.0], RecommendedItem[item:2700, value:5.0], RecommendedItem[item:1223, value:5.0], RecommendedItem[item:333, value:5.0], RecommendedItem[item:34, value:5.0], RecommendedItem[item:2565, value:5.0], RecommendedItem[item:2571, value:5.0], RecommendedItem[item:58, value:5.0], RecommendedItem[item:50, value:5.0]]
------------------ userid: 5------------------
[2028, 1259, 590, 858, 2728, 318, 296, 3334, 3481, 2772]
[6, 24, 39, 41, 47, 52, 224, 272, 288, 296, 318, 321, 356, 377, 497, 509, 515, 800, 866, 896, 913, 994, 1127, 1171, 1175, 1192, 1243, 1279, 1392, 1449, 1466, 1500, 1529, 1535, 1610, 1643, 1649, 1650, 1683, 1684, 1715, 1719, 1729, 1730, 1734, 1747, 1788, 1909, 1912, 1966, 2028, 2058, 2188, 2268, 2318, 2337, 2355, 2359, 2390, 2395, 2427, 2428, 2580, 2683, 2700, 2716, 2721, 2723, 2762, 2858, 2952, 2987, 2997, 3006, 3016, 3020, 3079, 3083, 3260, 3386, 3408, 3409, 3476, 3513, 3578, 3624, 3744, 3786, 3793, 3799]
************ current hits: 3 *****************
=============== precision: 0.1 =================
[RecommendedItem[item:2028, value:5.0], RecommendedItem[item:1259, value:5.0], RecommendedItem[item:590, value:5.0], RecommendedItem[item:858, value:5.0], RecommendedItem[item:2728, value:5.0], RecommendedItem[item:318, value:5.0], RecommendedItem[item:296, value:5.0], RecommendedItem[item:3334, value:5.0], RecommendedItem[item:3481, value:5.0], RecommendedItem[item:2772, value:5.0]]
------------------ userid: 6------------------
[3462, 1278, 1219, 1225, 1207, 1079, 2137, 3159, 50, 3147]
[199, 266, 368, 377, 383, 588, 595, 597, 920, 1028, 1035, 1101, 1188, 1210, 1296, 1441, 1674, 1947, 1959, 2017, 2100, 2406, 2506, 2802, 2858, 3074, 3501, 3508, 3600, 3624, 3699]
=============== precision: 0.083333336 =================
[RecommendedItem[item:3462, value:5.0], RecommendedItem[item:1278, value:5.0], RecommendedItem[item:1219, value:5.0], RecommendedItem[item:1225, value:5.0], RecommendedItem[item:1207, value:5.0], RecommendedItem[item:1079, value:5.0], RecommendedItem[item:2137, value:5.0], RecommendedItem[item:3159, value:5.0], RecommendedItem[item:50, value:5.0], RecommendedItem[item:3147, value:5.0]]
------------------ userid: 7------------------
[899, 837, 2959, 471, 2985, 2662, 2912, 2692, 306, 1204]
[6, 110, 349, 457, 474, 480, 589, 1221, 1270, 1573, 1610, 1722, 1997, 2028, 2571, 2916, 3256, 3578, 3753]
=============== precision: 0.071428575 =================
[RecommendedItem[item:899, value:5.0], RecommendedItem[item:837, value:5.0], RecommendedItem[item:2959, value:5.0], RecommendedItem[item:471, value:5.0], RecommendedItem[item:2985, value:5.0], RecommendedItem[item:2662, value:5.0], RecommendedItem[item:2912, value:5.0], RecommendedItem[item:2692, value:5.0], RecommendedItem[item:306, value:5.0], RecommendedItem[item:1204, value:5.0]]
------------------ userid: 8------------------
[912, 928, 858, 2028, 1197, 527, 1240, 3624, 3793, 3751]
[14, 16, 17, 24, 25, 39, 42, 110, 111, 150, 151, 161, 163, 230, 253, 265, 269, 288, 296, 337, 345, 377, 506, 508, 527, 538, 555, 589, 741, 1059, 1120, 1210, 1213, 1277, 1357, 1393, 1476, 1573, 1580, 1589, 1621, 1639, 1660, 1673, 1678, 1682, 1693, 1704, 1711, 1730, 1916, 2006, 2028, 2278, 2297, 2320, 2396, 2427, 2429, 2571, 2600, 2688, 2692, 2699, 2712, 2858, 3105, 3107, 3147, 3155, 3213, 3246, 3250, 3267, 3386, 3425, 3481]
************ current hits: 2 *****************
=============== precision: 0.0875 =================
[RecommendedItem[item:912, value:5.0], RecommendedItem[item:928, value:5.0], RecommendedItem[item:858, value:5.0], RecommendedItem[item:2028, value:5.0], RecommendedItem[item:1197, value:5.0], RecommendedItem[item:527, value:4.8], RecommendedItem[item:1240, value:4.6666665], RecommendedItem[item:3624, value:4.6666665], RecommendedItem[item:3793, value:4.6666665], RecommendedItem[item:3751, value:4.6666665]]
------------------ userid: 9------------------
[2542, 1997, 589, 1250, 1193, 1732, 2804, 1197, 1350, 50]
[1, 25, 50, 223, 318, 349, 367, 377, 412, 428, 457, 508, 529, 590, 593, 778, 805, 920, 1060, 1089, 1210, 1223, 1265, 1307, 1343, 1393, 1466, 1552, 1639, 1682, 1721, 1784, 1921, 1961, 2028, 2268, 2302, 3006, 3147, 3148, 3253, 3255, 3298, 3484, 3510, 3751, 3793]
************ current hits: 1 *****************
=============== precision: 0.08888889 =================
[RecommendedItem[item:2542, value:5.0], RecommendedItem[item:1997, value:5.0], RecommendedItem[item:589, value:5.0], RecommendedItem[item:1250, value:5.0], RecommendedItem[item:1193, value:5.0], RecommendedItem[item:1732, value:5.0], RecommendedItem[item:2804, value:4.6666665], RecommendedItem[item:1197, value:4.5], RecommendedItem[item:1350, value:4.5], RecommendedItem[item:50, value:4.5]]
------------------ userid: 10------------------
[318, 1089, 3148, 3421, 1721, 2384, 1961, 1198, 608, 246]
[1, 7, 24, 32, 62, 150, 153, 180, 186, 203, 223, 275, 277, 292, 316, 317, 318, 329, 333, 351, 356, 367, 435, 480, 520, 541, 542, 543, 551, 586, 587, 595, 596, 648, 653, 671, 745, 750, 784, 802, 858, 898, 899, 904, 916, 918, 923, 938, 953, 1015, 1035, 1059, 1079, 1084, 1091, 1104, 1127, 1135, 1136, 1148, 1196, 1197, 1198, 1200, 1220, 1221, 1230, 1240, 1250, 1252, 1254, 1256, 1282, 1283, 1291, 1293, 1302, 1304, 1307, 1320, 1339, 1356, 1380, 1394, 1411, 1544, 1566, 1573, 1586, 1654, 1682, 1721, 1756, 1884, 1918, 1921, 1927, 1948, 1961, 1967, 2005, 2021, 2040, 2046, 2047, 2072, 2080, 2082, 2090, 2093, 2096, 2108, 2111, 2124, 2133, 2138, 2161, 2174, 2193, 2294, 2300, 2336, 2355, 2371, 2378, 2398, 2405, 2407, 2423, 2431, 2436, 2470, 2497, 2501, 2529, 2530, 2599, 2617, 2622, 2628, 2640, 2657, 2693, 2717, 2746, 2762, 2770, 2791, 2795, 2804, 2872, 2918, 2946, 2948, 2959, 2968, 2997, 3035, 3037, 3066, 3095, 3114, 3155, 3174, 3175, 3194, 3254, 3309, 3358, 3359, 3438, 3489, 3500, 3591, 3668, 3675, 3699, 3701, 3702, 3703, 3723, 3751, 3809, 3812, 3869, 3928]
************ current hits: 4 *****************
Mahout POM file (partial)
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.mahout</groupId>
<artifactId>mahout</artifactId>
<version>0.8-SNAPSHOT</version>
<parent>
<groupId>org.apache</groupId>
<artifactId>apache</artifactId>
<version>9</version>
</parent>
<packaging>pom</packaging>
<name>Apache Mahout</name>
<url>http://mahout.apache.org</url>
<inceptionYear>2008</inceptionYear>
<description>
</description>
<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>
<organization>
<name>The Apache Software Foundation</name>
<url>http://www.apache.org/</url>
</organization>
<properties>
<skipTests>false</skipTests>
<maven.clover.multiproject>true</maven.clover.multiproject>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<hadoop.version>1.1.2</hadoop.version>
<lucene.version>4.2.0</lucene.version>
</properties>
<issueManagement>
<system>Jira</system>
<url>https://issues.apache.org/jira/browse/MAHOUT</url>
</issueManagement>
<repositories>
<repository>
<id>apache.snapshots</id>
<name>Apache Snapshot Repository</name>
<url>http://repository.apache.org/snapshots</url>
<releases>
<enabled>false</enabled>
</releases>
</repository>
</repositories>
</project>

How to save values of multidimensional array to db in Rails

I have a bit multidimensional array like this one:
[[2, 1], [751, 159], [793, 195], [978, 527], [938, 783], [977, 881], [713, 592, 101], [337, 248, 143], [444, 647, 196], [913, 763, 211], [752, 746, 225], [862, 878, 373], [876, 900, 597], [719, 904, 236, 11], [448, 836, 883, 59], [486, 723, 241, 150], [864, 611, 619, 167], [558, 796, 707, 353], [915, 683, 738, 534], [641, 777, 892, 107, 78], [264, 310, 161, 574, 114], [549, 849, 596, 181, 160], [633, 471, 497, 438, 262], [351, 485, 544, 772, 348], [890, 941, 572, 418, 374], [776, 972, 1001, 729, 379], [595, 898, 989, 771, 454], [84, 514, 914, 571, 442, 19], [335, 97, 689, 314, 879, 20], [869, 828, 802, 859, 553, 34], [298, 134, 564, 614, 411, 62], [175, 579, 490, 399, 479, 72], [382, 830, 215, 691, 624, 85], [922, 840, 657, 917, 436, 95]]
As you can see, each sub array has between 2 and 6 values.
I have a model called "Sub" with db columns "sub1", "sub2", "sub3", "sub4", "sub5", "sub6". (values for sub3-6 would be null if the sub array only had 2 values)
What is the proper way to save each sub array to a db row?
I have tried this:
#m = [the big array I mentioned above]
#m.each do |t|
Pair.create(:sub1 => t[0], :sub2 => t[1], :sub3 => t[2], :sub4 => t[3], :sub5 => t[4], :sub6 => t[5])
end
But for some reason that creates a table that does this: (sample of "sub1" column)
2
2
751
2
751
793
2
751
793
978
2
751
793
978
938
See what's happening there? It repeats each but adds one more row each time.
Here's my model:
class Pair < ActiveRecord::Base
attr_accessible :id, :sub1, :sub2, :sub3, :sub4, :sub5, :sub6
end
UPDATE: I figured out the issue... I had the create method in the wrong part of a script and that was causing it to run several times in a row in a weird way. Thanks everyone!
One way would be to serialize the column. See here: http://api.rubyonrails.org/classes/ActiveRecord/Base.html#method-c-serialize at the Saving arrays, hashes, and other non-mappable objects in text columns section.

Resources